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20.  (Continued) 

identify  any  permanence  in  effects  thereby  limiting  results  to 
indications  of  predictive  performance.  A  management  aid  is 
developed  consisting  of  data  base  and  manipulative  APL  programs 
for  the  purposes  of  updating  and  displaying  current  information 
while  identifying  possible  synergistic  interactions  between 
data  subcatagories .  Appendix  A  contains  the  Users  Guide  for 
this  management  aid.  These  results  can  be  applied  to  the 
current  forecast  to  aid  the  department  chairman  in  creating 
the  faculty  teaching  schedule. 
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ABSTRACT 


The  course  loading  forecasts  for  the  Operations  Research 
Department  published  by  the  Naval  Postgraduate  School 
Programming  Office  are  investigated  for  measures  of 
predictive  accuracy  by  course  and  curricula.  Both  course 
and  curricula  forecasts  are  categorized  as  problem  causing, 
reasonably  accurate,  or  undefined  due  to  limited  informa¬ 
tion.  The  paucity  of  the  data  precludes  the  use  of  sophis¬ 
ticated  statistical  techniques  to  identify  any  permanence  in 
effects  thereby  limiting  results  to  indications  of 
predictive  performance.  A  management  aid  is  developed 
consisting  of  a  data  base  and  manipulative  APL  programs  for 
the  purposes  of  updating  and  displaying  current  information 
while  identifying  possible  synergistic  interactions  between 
data  subcatagories.  Appendix  A  contains  the  Users  Guide  for 
this  management  aid.  These  results  can  be  applied  to  the 
current  forecast  to  aid  the  department  chairman  in  creating 
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I.  BACKGROUND 


T 


A.  PROBLEM  HISTORY 

.he  Naval  Postgraduate  School  uses  a  complex  prediction  • 

system  to  forecast  expected  student  demand  for  courses  over 
the  upcoming  year.  This  forecast  is  computed  using  various 
inputs  from  several  sources  during  the  summer  quarter  (July 
to  September)  and  covers  the  following  academic  year  -  fall 
through  summer  quarters  inclusive.  The  tabled  output  of 
course  loading  predictions  so  computed  is  used  by  the 
various  Department  Chairmen  to  schedule  faculty  teaching 
responsibilities  and  research  time  for  that  year.  As  with 
any  forecasting  system,  this  set  of  course  loading  predici- 
tons  is  not  exact.  Various  errors  are  introduced  either 
from  the  simplification  of  the  process  required  to  get  the 
model  to  a  managable  size  or  from  erroneous  input.  The 
Operations  Research  Department  Chairman  noticed  an  inaccu¬ 
racy  problem  when  a  class  with  a  predicted  size  of  approxi¬ 
mately  30  was  scheduled  and  approximately  75  students  showed 
up,  causing  a  disruption  of  faculty  teaching  plans. 

Believing  that  a  problem  existed,  he  posted  a  request  to 
research  the  scope  of  the  inaccuracies  as  pertaining  to  the 
courses  taught  by  the  Operations  Research  Department.  The 
study’s  goal  would  be  to  determine  if  a  significant  problem 
existed  and,  if  so,  to  provide  either  a  series  of  adjust¬ 
ments  to  be  made  tc  the  present  predictions  or  a  better 
prediction  method. 

B.  PRESENT  SYSTEM 

Until  1972,  the  scheduling  and  course  prediction  system 
was  run  by  hand  using  "corporate  memory";  the  historical 
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demand  for  courses  as  recalled  by  the  curriculum  representa¬ 
tives  involved.  These  recollections  were  used  to  prepare  a 
proposed  schedule  that  the  students  and  curricular  offices 
used  and  modified  as  required.  By  1972,  this  system  had 
just  about  reached  the  limits  of  its  capabilities.  At  that 
time,  the  scheduling  problem  was  passed  on  to  a  newly  hired 
NPS  computer  program  designer1  who  tailored  a  predictive 
computer  model  to  the  school's  needs.  That  model,  with 
modifications,  is  still  in  use  today. 

1 .  Scheduling  Z  Prediction  Problems 

The  problem  faced  in  1971  is  the  same  today  -  How 
can  we  accurately  predict  three  items: 

•  The  courses  required  to  match  student  demand 

•  When  those  courses  should  be  taught 

•  How  many  students  will  attend 

Two  types  of  student  inputs  are  available;  those  who  are  not 
yet  onboard  (with  or  without  orders) ,  and  those  presently 
attending  the  school  in  any  given  capacity.  Both  types  of 
students  are  designated  as  attending  a  given  curriculum,  and 
from  that  designation  a  standard  program  of  courses  is 
assigned.  This  assignment  leads  to  the  course  scheduling 
output.  Since  the  two  input  groups  are  significantly 

different  with  respect  to  treatment  within  the  model  itself, 
they  will  be  separately  discussed. 

a.  Students  not  yet  onboard 

The  Office  of  the  Chief  of  Naval  Operations 
publishes  a  yearly  qucta  plan,  [Ref.  1],  which  directs  the 
Commander,  Naval  Military  Personnel  Command  to  provide  the 
officer  personnel  to  fill  the  allocated  graduate  education 
hillets.  The  fiscal  year  1985  juota  summary  is  shown  in 


1Mike  Troian,  Special  Assistant  for  Programs: 
working  under  Code  0  13:  J.  Barron,  Capt.  USN  (ret) 


presently 


table  I  .  These  quotas  are  further  broken  down  to  show,  by 
professional  designator  (i.e.  Surface  Warfare,  UFO, 
Cryptology  .  .  .) ,  what  curricula  the  officers  will  attend. 
Historically,  however,  these  guotas  have  been  guidelines  and 
not  strictly  followed.  Additionally,  Naval  Military 
Personnel  Command  (NMPC)  gives  only  a  rough  idea  of  in  which 
quarter  an  officer  will  arrive  during  a  given  year.  This  in 
turn  drives  the  input  problem  experienced  at  NPS.  Through 
liason  with  NMPC,  the  school  identifies  as  specifically  as 
possible  just  who  has  orders  to  attend  NPS  either  in  hand  or 
being  written.  Quota  shortfalls  tend  to  occur  in  the  "tech¬ 
nical"  curricula  due  to  a  lack  of  available,  qualified 
personnel.  Conversly,  surpluses  occur  in  the  "non¬ 
technical"  curricula.  As  with  any  military  organization, 
operational  commitments  such  as  maintaining  ships  in  the 
Eastern  Kediterranian  Sea  or  Indian  Ocean  adversly  affect 
the  arrival  of  Surface  Warfare  Officers  (who  consitute  the 
largest  group  in  the  NPS  student  population) . 

The  school  folds  all  of  this  guota  information 
and  experience  into  a  "best  guess"  of  the  number  of  students 
arriving  per  curriculum  per  goarter.2  As  this  process  is 
being  completed,  the  curricular  officers  designate  the  paths 
these  new  arrivals  will  follow.  This  is  done  either  by 
service  (i.e.  USN  students  take  the  following  courses:  . 

.)  or  by  past  experience  (i.e.  28%  of  incoming  students  need 
refresher  math  followed  by  .  .  .).  The  resulting  linking  of 
numbers  of  students  and  assigned  courses  provides  one  type 
of  input  to  the  scheduling  model. 


2 Additionally.  the  programming  office  contacts  indi¬ 
vidual  service  and  foreign  representatives  to  further  refine 
the  expected  inputs  (since  [Ref.  1]  pertains  onlv  to  USN 
officers).  These  organizations  provide  less  reliable  data 
mainly  because  the  fiscal  year  for  other  US  services  starts 
in  January.  The  planning  for  their  personnel  transfers  is 
therefore  not  complete  until  October  or  November. 


TABLE  I 

Summary  table  from  [Ref.  1] 


t.  Students  presently  attending  NPS 


The  scheduling  process  for  those  students  • 

already  onboard  is  scmewhat  simpler.  Since  they  are  already 
designated  as  attending  a  given  curriculum  over  a  known 
period  of  time,  the  only  area  in  which  significant  problems 
are  found  is  that  of  elective  scheduling.  Since  the  • 

prediction  is  completed  a  full  year  in  advance,  out-guarter 
electives  may  not  have  been  correctly  selected  by  the 
student.  His  interests  or  his  required  course  load  may 

change,  thereby  changing  any  electives  he  has  selected.  • 

Additionally,  the  course  demand  may  be  such  that  the  course 
is  withdrawn  from  the  list  of  available  courses.  In  all 
cases,  however,  course  selection  input  covering  the  upcoming 
year  is  made  by  the  student  through  his  curricular  officer  • 

to  the  scheduling  office. 

2.  Scheduling  /  Prediction  System  Ose 

Once  elective  selections  have  been  made  by  the  » 

onboard  students  and  the  numbers  of  new  inputs  per  curric¬ 
ulum  are  entered,  the  course  loading  prediction  model 
applies  standard  course  matrixes  to  generate  the  overall 
student  course  demand.  The  scheduling  system  is  demand  '• 

based,  meaning  this  derived  demand  drives  the  faculty  hiring 
and  scheduling  process.  The  budget  process  also  is  based  on 
the  course  demand.  Teaching  funds  are  allocated  for  all 
instructors  teaching  course  sections  with  five  or  more  • 

students.  The  individual  department  chairmen  receive  a 
listing  cf  student  demands  by  course  and  quarter  from  which 
they  create  a  proposed  schedule  of  courses  to  be  offered. 

This  schedule  is  further  adjusted  through  the  allocation  of  • 

teaching  funds  in  a  process  consisting  of  an  approximately 
month  long  dialog  between  the  department  chairmen,  division 
deans,  and  administrative  representatives  from  the 
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programming  office.  The  goal  of  this  process  is  to  create  an 
acceptable  schedule  of  courses  to  be  offered  balanced 
between  annual  operating  budget  funds,  student  course 
demands  and  faculty  research  reguirements.  Through  this 
scheduling  process,  some  courses  are  dropped  and  others 
added,  creating  some  student  movement  among  the  courses 
offered.  The  goal  of  the  course  loading  prediction  model  is 
to  minimize  the  course  changes  caused  by  this  scheduling. 3 


3Hike  Troian,  the  Special  Assistant  for  Programs  working 
from  Ingersol  Room  375,  has  routinely  conducted  accuracy 
reviews  of  the  course  loading  prediction  model.  For  FYT983. 
this  review  has  shown  a  90$  accuracy  in  predicting  actual 
courses  taught  with  a  less  than  1%  deviation  from  total 
number  of  courses  predicted  (his  figures).  As  an  indicator 
of  faculty  required  and  expected  funding  costs,  the  model  is 
reportedly  excellent. 


II.  INITIAL  STEPS  AT  ANALYSIS 

Before  any  fact-based  discussion  of  "better”  predictions 
could  be  started,  the  problem  facing  the  Department  Chairman 
had  to  be  defined.  A  single  historical  observation  of  a 
severe  prediction  error  that  caused  problems  is  not  signifi¬ 
cant  considering  the  cumber  of  predictions  and  observations 
made.  Historical  data  had  to  be  checked  to  determine  if  the 
problem  cf  inaccurate  predictions  actually  existed  or  was 
merely  assumed  based  on  a  seldom  occurring  event.  Since 
predictions  were  made  by  course,  the  obvious  choice  for 
comparison  was  the  historical  course  data  available  from  the 
registrar’s  office.  Not  until  these  steps  were  completed 
could  a  search  for  a  tetter  method  of  prediction  be  started. 

A.  EATA  ORGAN IZ ATIO  1 

One  cf  the  initial  steps  of  the  analysis  was  an  organi¬ 
zation  of  all  the  usable  data  into  a  quantifiable  format. 
Useful  information  available  consisted  of  the  following 
items : 

•course  number 

•quarter  taught 

•number  of  students  predicted 

•number  of  students  attending 

The  eventual  goal  of  the  analysis  became  that  of  finding 
such  permanencies  in  the  data  (based  on  these  items)  as  to 
provide  an  accurate  prediction  adjustment. 

1 .  Predic  >n  Data 


Prediction  data  was  available  both  for  individual  curricula 
inputs  into  a  given  course  and  for  the  total  expected 


student  demand  for  those  courses.  This  data  was  presented 
in  two  formats:  summary  data  and  individual  course  data. 
The  summary  data  (figure  2. 1)  existed  for  academic  years 
1982,  1S83  and  the  present.  This  data  provided  a  given 
years  display  of  the  total  expected  student  loading  indexed 
by  course  and  quarter.  It  included  a  recommended  number  of 
"sections  taught"  and  the  corresponding  faculty  hours 
required.  As  seen  in  the  figure,  the  predictions  are  made 
for  a  given  academic  year,  in  this  case  84  (September  1983 
to  August  1984) .  The  first  column  contains  the  alpha¬ 
numeric  course  identifier  as  listed  in  [Ref.  2].  The  'ST'J' 
columns  refer  to  the  number  of  students  predicted  to  attend, 
the  'SEC1  column  refers  to  the  number  of  sections  required 
to  accomodate  those  students  (maximum  section  size  is 
limited  to  30  attendees),  and  the  'FAC-HRS'  columns  refer  to 
the  number  of  faculty  teaching  hours  needed  to  teach  those 
sections. 
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For  1983  and  1984,  individual  coarse  predictions 
were  available  (figure  2.2).  These  coarse  predictions  listed 
the  expected  student  loading  for  a  given  course  during  a 
given  guarter  coded  by  degree  awarded.  This  degree  coding 
is  found  in  the  "S/D/T"  (specialty/degree/time)  column  of 
the  data.  The  first  two  letters  of  this  column  apply  to  the 
curriculum  specialty,  the  next  two  to  the  degree  (MS  refers 
to  Master  of  Science).  This  coding  is  easily  tr ansf errable 
to  curriculum  number  using  Naval  Postgraduate  School 
instruction  1520.19  (figure  2.3).  The  numbers  that  occa¬ 
sionally  appear  following  those  four  letters  indicate  when 
curricular  officers  have  not  provided  an  updated  list  of 
whom  they  expect  for  that  course  at  that  time  in  that 
specialty.  The  numbers  mean  that  the  prediction  listed  came 
from  the  "standard  files"  of  the  programming  office;  a 
collection  of  best  guesses  of  the  expected  number  of  atten¬ 
dees  loosely  based  on  historical  patterns.*  These  individual 
course  predictions,  however,  make  no  mention  of  segment 
assignments. 


♦ether  columns  in  the  individual  course  prediction  data 
are  generally  self-explanatory  with  the  exception  of  these 
minor  items:  The  AV  column  referred  to  the  number  of  avia¬ 
tors  (Pilots,  NFOs,  etc.)  that  attend  and  is  an  unused  relic 
of  some  past  program.  The  letters  sometimes  preceding  the 
section . column  are  now  meaningless.  The  section  column  code 
itself  is  assigned  by  the  curricular  offices  to  groups  of 
incoming  students. 


Example  of  Individual  Course  Predictions 
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NAVPGSCOLINST  1520.19  CH-1 
21  October  1983 


CODES  OSED  FOR  SCHEDULING 

- A5~OF - 

JUNE  1983 


CODE 

30 

SPECIALTY 

DEGREE 

CODE  SPECIALTY 

360 

RAMS 

RGMS 

RNMS 

RSMS 

RPPH 

RC  MS 

OR/SA  ARMY 

OR/SA  GENERAL 

OR/SA  NAVY 

OR/SA  SUPPLY  CORPS 

OR/SA  PHD  IN  OR 

GRCCP  OF  ANY  OF  ABOVE 

CODE 

31 

610 

AC  MS 

AERONAUTICAL  ENGINEERING 

611 

AXMS 

ATMS 

AERONAUTICAL  ENGINEERING  (Avionics) 

AERO  ENGINEERING  FOR  TEST  PILOTS 

CODE 

32 

600 

DC  MS 

ELECTRICAL  ENGINEERING  (Communications) 

590 

EAMS 

ELECTRICAL  ENGINEERING 

620 

HCMS 

HMMS 

COMMUNICATION  MANAGEMENT  (Coast  Guard) 
COMMUNICATION  MANAGEMENT 

595 

KEMS 

ELECTRONIC  WARFARE  SYSTEMS 

366 

SOMS 

SPACE  SYSTEMS  OPERATIONS 

591 

SEMS 

SPACE  SYSTEMS  ENGINEERING 

CODE 

33 

380 

GM  MS 

MATH  OPTION  OF  ADVANCED  SCIENCE 

381 

GPMS 

PHYSICS  OPTION  OF  ADVANCED  SCIENCE 

530 

WTMS 

WEAPONS  SYSTEM  ENGINEERING 

531 

WSMS 

WEAPONS  SYSTEM  SCIENCE 

532 

WNMS 

NUCIEAR  PHYSICS  (Weapons  and  Effects) 

535 

UXMS 

UNDERWATER  ACOUSTICS 

CODE 

331 

525 

IXMS 

ANTISUBMARINE  WARFARE 

Figure  2.3  Partial  Listing  of  Degree  to  Curriculun  Codes 
(see  [Ref.  3]  or  Appendix  B  for  complete  listing). 


2. 


Historical  Data 

Historical  attendance  data  was  made  availatle  by  the 
registrar’s  office.  This  data,  filed  by  quarter5  listed  the 
courses  taught  in  alpha-numeric  order  displaying  student 
names,  curricula,  and  grades  received  (see  figure  2.4).  The 
numbers  recorded  reflect  the  number  of  students  completing 
the  course  vice  the  number  seated  on  the  first  day  of  class. 
Segements  are  listed  separatly.  Use  of  this  historical  data 
implies  a  number  of  actions.  First,  the  predictions  are 
demand  based  and  would  probably  correlate  better  with  the 
number  of  students  registered  for  the  course  than  with  the 
number  completing.  A  strong  positive  correlation  is  assumed 
between  number  of  students  registered  and  number  completing 
a  given  course.  Second,  grade  and  name  information  was 
discarded  thereby  burying  any  measure  of  correlation  between 
performance  and  attrition.  Third,  segment  data  was  collapsed 
to  match  individual  course  prediction  data.  This  obscured 
relationships  between  predicted  and  actual  number  of 
sections  taught  or  faculty  hours  required.  Since  the  goal  of 
the  analysis  was  to  adjust  the  predictions  of  student  demand 
vice  to  explain  the  reasons  an  adjustment  is  necessary, 
these  actions  were  considered  acceptable. 


5The  registrar’s  system  is  rather  unique  in  its  ceding 
of  this  data.  ’School  Year  84’  does  not  refer  to  fiscal 
year  84  (normally  1  October  1983  -  3TJ  September  1984). 
Instead,  'School  Year  84'  refers  to  the  September  84  to 
August  85  quarters.  As  this  data  was  collected,  the  coding 
of  quarterly  data  (to  be  discussed  later)  was  translated 
fcacR  intc  standard  fiscal  year  notation.  Thus,  throughout 
the  data  base  used  in  the  analysis,  83.1  refers  to  the 
September  82  -  December  82  quarter  of  the  1982  -  1983  school 
year,  as  expected. 
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OFFICIAL  GRADE  ROS 

TER 

PAG  E  1 

NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY, CALIFORNIA 

93943 

DATE  13  APR 

COURSE  NO. 

OS  36  02 

SEGEMENT 

00  — 

QUARTER  1 

AY  83-84 

COMBAT 

MODELS 

PROF. 

SHUDDE,  REX  H. 

3  LE 

CTURE 

HCURS  1  LABORATORY 

HOURS 

CORPS/ 

NAME 

RANK 

COUNTRY 

CURR 

MARK 

1. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#374 

A 

2. 

xxxxxx. 

xxxx 

X 

LCD  R 

US  N 

#374 

B 

3. 

xxxxxx. 

xxxx 

X 

LCD  R 

USN 

#525 

3 

4. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

5. 

xxxxxx. 

xxxx 

X 

LCD  R 

USN 

#525 

A- 

6. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

A 

7. 

xxxxxx. 

xxxx 

X 

LCD  R 

USN 

#374 

A- 

8. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

A 

9. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

3 

10. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

11. 

xxxxxx. 

xxxx 

X 

LCD  R 

USN 

#525 

B 

12. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

3 

FORM 

4 

THIS 

IS  THE 

OFFICIAL 

GRADE 

ROSTER 

TO 

BE  FIL 

ED  IN  THE 

ACADEMIC  DEPARTMENT 

Figure  2.4  Sample  Attendance  Data  (names  removed) 

3*  Coding  System 

Before  entering  the  data  into  computer  files,  a 
coding  system  was  devised  to  distill  the  raw  data  into  its 
usable  parts.  Excess  information  such  as  student  names  and 
grades  could  then  be  discarded.  The  data  consisted  of 
predictions  and  actual  attendance  figures  coupled  with  iden¬ 
tifiers.  Identifiers  required  include  course  number  and 
quarter  taught.  Available  data  included  both  total  and  indi¬ 
vidual  curricula  figures.  For  simplicity  of  use,  a  basic 
file  was  created  using  the  following  five-column  coding 
scheme: 


Quarter  Curricula#  Course#  #attended  #predicted 


Once  these  items  had  been  tabulated,  an  easy  subtraction 
(attending  minus  predicted)  provided  an  error  term.  By 
ordering  (by  magnititude  of  error)  and  displaying  the 
resulting  array,  a  history  of  many  poor  predictions  was 
observed.  This  observation  confirmed  that  a  problem  existed 
and  showed  a  need  for  further  analysis. 

B.  NATURAL  DIVISIONS  IN  CODED  DATA 

The  large  volume  of  coded  data  created  was  divisible 
into  several  sub-categories.  The  Operations  Research 
Department  teaches  two  types  of  courses:  OA  -  taken  by  oper¬ 
ations  analysis  major  students;  and  OS  -  taken  by  students 
from  ether  curricula.  Initially  both  of  these  types  were 
taken  under  consideration  for  analysis.  Later,  the  OA 
designated  courses  were  dropped  from  consideration  for  two 
reasons;  the  problem  of  not  knowing  how  many  students  were 
going  tc  show  up  was  generally  solvable  within  the  depart¬ 
ment;  additionally,  time  constraints  limited  the  scope  of 
the  areas  to  be  covered.6  The  OS  course  data  was  broken  into 
two  prediction  areas:  summary  predictions  and  curricula 
specific  predictions.  The  summary  predictions  were  used  to 
show  patterns  by  course;  the  curricula  predictions  to  show 
patterns  by  individual  curricular  offices. 

1 .  Summary  Data 

When  dealing  with  summary  prediction  data  error 
terms,  the  goal  of  the  analysis  was  to  provide  a  functional 
relationship  between  predictions  and  attendance  by  course. 


6CJ)  data  collected  during  the  initial  stages  of  problem 
investigation  is  available.  The  format  is  slightlv 
different  from  that  of  the  OS  course  data  because  the 

froblem  found  for  OA  dealt  with  elective  selection.  The 
actcr  of  interest  was  service  affiliation.  For  further 
details,  see  appendix  A. 
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Using  these  derived  relationships  (hopefully  simple  addi¬ 
tions  or  subtractions) ,  the  department  chairman  will  be  able 
to  take  the  forecast  from  the  programming  office  (figure 
2.1)  and  change  the  figures  shown  to  reflect  more  accurately 
the  expected  student  load.  These  more  accurate  figures  can 
then  be  used  to  schedule  faculty  teaching  assignments, 
request  necessary  funding,  and  accomplish  other  tasks 
requiring  the  number  cf  expected  students  per  course. 

2.  Curricula  Specific  Data 

Rhen  dealing  with  error  terms  for  individual 
curricula,  the  goal  of  the  analysis  was  to  provide  a  func¬ 
tional  relationship  between  predictions  and  attendance  by 
curriculum.  Using  these  derived  relationships,  the  depart¬ 
ment  chairman  will  be  able  to  take  the  forecast  for  an  indi¬ 
vidual  course  (figure  2.2)  and  change  the  figures  shewn  to 
reflect  more  accurately  the  expected  student  load  by  either 
applying  the  relationship  or  calling  the  curricular  office 
involved  to  get  better  numbers.  These  historical  observa¬ 
tions  will  enable  the  department  chairman  both  to  identify 
curricula  that  are  inadvertently  causing  scheduling  problems 
and  to  show  improvements  or  deterioration  in  the  scheduling 
process  itself.  This  information  can  be  used  both  by  the 
programming  office  and  the  curricular  office  involved  as 
well  as  the  OR  Department. 


III.  DETAILED  ANALYSIS  OF  OS  PREDICTIVE  DATA 


At  the  completion  of  the  initial  steps  toward  analysis 
previously  discussed,  a  coded  series  of  data  arrays  were 
available  for  use.  An  example  of  array  formatting  is  shown  • 

in  figure  3.1  The  figure  of  interest  was  the  "error"  term 
found  in  column  six.  These  error  terms,  representing  the 
difference  between  the  actual  number  of  students  attending  a 
course  vs  the  number  predicted,  were  displayed  in  various  • 

groups  ordered  by  magnitude  to  show  the  accuracy  patterns  of 
the  forecast.  Areas  that  were  explored  for  effects  included 

the  curriculum  that  provided  the  input,  the  courses  them¬ 
selves,  and  the  guarter  for  which  the  courses  were  I 

predicted.  Underlying  patterns  such  as  an  individuals 
"path"  through  various  OS  courses  required  for  his  curric¬ 
ulum  were  also  investigated.  The  paucity  of  the  data 

precluded  the  use  of  sophisticated  statistical  techniques  i 

thereby  limiting  results  to  indications  of  areas  of  concern 
rather  than  definitive  statistical  statements. 

A.  THE  DATA  i 

Six  element  data  arrays  (as  shown  in  figure  3.  1)  were 
created  for  each  years  data.  These  APL  variables  were 

combined  to  form  a  single  compilation  of  all  the  available 

i 

data  (see  table  II)  .  The  error  terms  of  this  compilation 
were  used  for  the  analysis.  These  error  terms  are  discrete 
integer  data  the  size  of  which  are  limited  by  the  maximum  of 
the  actual  or  predicted  columns  terms.  Distributional 

I 

fitting  was  not  done  for  two  reasons;  the  population  should 
have  high  internal  correlation  due  to  the  interrelation 
between  course  errors  caused  by  student  course  matrices,  and 


QTE 

*1  ** 

CUKRIC 

****** 

CSE 

*** 

ACT 

*** 

PRE 

*** 

(ACT-PRE) 

********* 

8  4.  3 

0 

2103 

0 

1 

-1 

84.3 

535 

210  3 

0 

1 

-1 

84.3 

0 

300  1 

25 

0 

25 

84.  3 

363 

3001 

25 

0 

25 

84.3 

0 

3002 

15 

18 

-3 

84.  3 

683 

3002 

0 

1 

-1 

84.3 

• 

825 

3002 

15 

17 

-2 

• 

83.  1 

0 

3602 

7 

7 

0 

83.  1 

374 

3602 

1 

1 

0 

83.  1 

525 

3602 

6 

6 

0 

83.  1 

0 

3603 

20 

21 

-1 

83.  1 

595 

3603 

20 

21 

-1 

83.  1 

0 

3604 

18 

16 

2 

83.  1 

365 

3604 

1 

0 

1 

QTE:  Quarter  taught.  83. 1  refers  to  the  first  guarter 
m  the  Sept.  82  to  Aug.  83  school  year. 

Currie:  Curricula  -  0  implys  summary  data  for  a  given 
course. 

Cse:  Course  number  (all  OS  courses). 

Act:  Actual  number  of  students  listed  on  final  grade 
records  as  completing  course. 

Pre:  Predicted  number  of  students  (from  forecast) . 

Act-Pre:  Prediction  error  term. 


Figure  3.1  Example  of  Array  Format  (with  definitions) 

API  variable  used  :  OS. 

the  sample  sizes  of  those  homogeneous  populations  distilled 
from  the  data  (such  as  "all  365  students  taking  OS  3604  in 
the  second  guarter")  were  prohibitively  small.  Two  types  of 
errors  were  discovered,  predictive  errors  and  accuracy 
errors.  A  predictive  error  was  defined  as  either  the  fore¬ 
cast  predicting  no  students  and  students  attending  or  vice 
versa.  Embedded  in  this  error  type  are  those  courses  which 
were  not  even  listed  but  taught  and  those  courses  dropped 
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TABLE  II 

APL  Variables  used  in  Analysis  | 

(see  Appendix  A  for  details  of  use)  j 

VARIAELE 

******** 

SHAPE 

******* 

DESCRIPTION 

************ 

OS82 

69x6 

AY  (Academic  year)  81-82  data; 
includes  summary  information  only 
(no  data  for  individual  curricula) 

OS83 

255x6 

AY  82-83  data;  complete  listing 

OSTRK83 

140x8 

AY  82-33  non-entry  course  data  * 

OS84 

228x6 

AY  83-84  data;  only  gtrs  1,2  and  3 

OS 

552x6 

Combination  of  all  of  above 
(without  repetition) 

*  see  ’Course/Curriculum  Interactions  and  Tests’  j 

section  for  complete  description.  j 

J 

from  the  schedule  in  the  budgeting  process.  The  same  divi¬ 
sion  was  found  by  individual  curricula  (for  example,  curric¬ 
ulum  460  (engineering  science  -  preparatory)  is  never 
predicted  in  any  individual  course  forecast)  .  Predictive 
errors  exist  when  columns  five  or  six  of  the  data  array 
contain  a  zero.  Accuracy  errors  are  other  differences 
between  the  actual  and  predicted  number  of  students. 

E.  CCUBSE/CURRICULtJH  INTERACTIONS  AND  TESTS 

Each  individual  student  at  the  Naval  Postgraduate  School 
follows  an  assigned  course  matrix.  This  assignment  varies 
by  both  the  program  assigned  and  individual  student  needs. 
However,  all  matrices  list  a  series  of  courses  that  are 
either  required  or  elecive.  The  Operations  Analysis  curric¬ 
ulum,  through  its  OS  course  offerings,  provides  a  service  to 
other  programs  by  providing  basic  and  advanced  pobability 
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and  statistics  training  not  available  in  other  curricula. 
These  service  courses  are  generally  required  by  ether 
curricula  as  a  prerequisite  to  both  further  training  and 
degree  completion.  As  such,  they  are  taught  in  a  certain 
sequence.  The  amount  of  training  required  differs  with 
different  programs,  so  various  OS  "tracks"  are  required. 
Sequencing  of  courses  in  this  manner  will  cause  dependence 
between  the  error  terms  of  certain  courses.  To  determine 
the  dependancy  relationship  within  the  data,  the  course 
matrices  for  all  curricula  offered  at  the  school  were 
collected  and  the  OS  course  sequences  extracted.  These 
sequences  were  combined  to  form  a  series  of  condensed  course 
matrices  as  applicable  to  the  Operations  Research  Department 
(see  figure  3.2  -  Note:  x1  s  refer  to  students  onboard  but 
not  taking  OS  or  OA  designated  courses) .  From  these 
matrices,  patterns  emerged.  As  with  any  program,  students 
started  with  basic  courses  and  moved  to  more  advanced 
topics.  Five  basic  courses  were  defined  as  "entry  level" 
courses.  One  of  these  courses  was  required  before  any 
advanced  work  was  done.  Since  the  entry  level  courses  were 
generally  taken  early  in  a  student’s  time  in  his  pregram, 
and  since  the  entry  course  was  the  first  time  he  appeared 
eligible  for  "prediction",  the  entry  level  courses  were 
expected  to  have  the  highest  variance  in  error  terms.  This 
proved  to  be  true.  As  the  students  spend  more  time  in  their 
programs,  attrition  and  schedule  changing  should  level  off. 
Additionally,  since  most  students  spend  more  than  a  single 
year  going  through  their  programs,  "down-line"  predictions 
will  be  improved  merely  by  the  students  moving  from  the  "not 
yet  onboard"  to  the  more  stable  "attending  NPS"  prediciton 
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Figure  3.2  Condensed  Course  Matrix  Examples 


Definition  of  entry  level  courses  led  to  the  definition 
of  course  sequence  "tracks".  A  student  from  a  given 
curricula  will  take  an  "entry"  course  -  the  first  time  he  is 
seen  ty  the  OA  Department  -  and  will  then  take  follow-on 
courses  tased  on  his  course  matrix.  Each  curriculum  has  its 


Quarter  Currie  Cse  Act  Pre  Dpre  (Act-Pre)  (Act-Dpre) 


Act  :  Actual  number  attending 
Pre  :  Standard  predictions 
Dpre  :  lerived  predictions 

A  display  of  the  results  showed  no  observable  difference 
between  the  regular  error  term  and  the  derived  error  term. 
A  flilcoxian  signed  rank  test7  was  done  to  test  the 
hypothesis  that  the  error  magnitude  of  the  adjusted 
predictions  tends  tc  be  smaller  than  the  magnitude  of  the 
error  from  the  forecast  (see  table  III) . 


TABLE  III 

Wilcoxian  Signed  Bank  Test  Resalts 

Sensitive  assumption:  Differences  independent. 
Assumed  large  amounts  of  varied  data  negated 
track  effects  resulting  in  relatively  independent 
difference  terms. 

CASE  A  CASE  B 

Variables  Tested:  X  =Curric  ACT-DPRE  Sum  ACT-DPRE 

Y  =Curric  ACT-PRE  Sum  ACT-PRE 

H  :  The  X  values  do  not  tend  to  be  smaller  than 
0  the  Y  values. 

H  :  The  X  values  tend  to  be  smaller  than 
1  the  Y  values. 

Decision  Rule  :  Reject  at  the  5%  confidence 

level  if  TS  >  1.65 


Test  Statistic  : 


CASE  A  CASE  3 
.453  -1.455 


Conclusion  :  Cannot  reject  null  hypothesis. 


7Taken  from  [Ref.  5  p.  280  ]. 
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The  results  of  the  test  indicated  that  the  adjusted 
predictions  were  nc  improvement  over  the  forecast.  That 
conclusion  indicates  that  interactions  more  complex  than  the 
following  of  course  matrixes  are  involved  in  actual  student 
movement  from  course  to  course.  The  adjusted  prediction 
method  was  dropped  from  the  analysis  as  a  means  of  better 
predicting  course  loading. 


u 


m 


¥ 


C.  01EEB  INTER ACTIO  US  AND  TESTS 

Interactions  exist  in  areas  separate  from  the  dependen¬ 
cies  created  within  a  class  sequence  by  the  course  matrix 
structure.  Three  main  effects  were  studied.  First, 
curricula  may  have  individual  patterns  across  time.  Seme 
groups  may  consistently  have  an  optimistic  input  that  causes 
an  overestimation  of  number  of  students  wherever  those 
groups  appear.  Second,  individual  courses  may  show  patterns 
of  over-  or  underestimation.  Lastly,  a  quarterly  effect  on 
errors  may  appear  due  strictly  to  the  elapsed  time  between 
the  prediction  and  the  actual  class.  To  provide  an  initial 
look  at  the  data  to  show  such  patterns,  two  programs  were 
written.®  These  programs  were  designed  to  sort  the  data  into 
the  catagories  of  interest,  specifically  curriculum,  course, 
and  quarter.  The  resulting  groupings  showed  that  there  were 
patterns  in  each  catagory,  but  that  the  number  of  observa¬ 
tions  available  to  base  conclusions  upon  made  those  patterns 
statistically  insignificant.  Through  clever  data  selection, 
a  series  of  observations  was  gleaned  to  test  for  curriculum 
and  course  effects  using  median  polish  technique  as  detailed 
in  [Ref.  4],  The  results  of  that  test  were  inconclusive 


®These  programs  and  others  used  in  the  analysis  are 
fully  explained  and  documented  in  appendix  A.  That  appendix 
also  contains  instructions  for  the  maintenance  and  use  of 
the  data  base  created,  and  procedures  for  gaining  access  to 
same. 
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showing  no  temporal  stability  (see  table  IV) .  The  test  for 
quarterly  effects  was  not  done  due  to  the  paucity  of  the 

data . 9  • 


^Complete  data  existed  only  tor  FY83.  This  limited 
sample  was  evaluated  as  too  unrepresentative  to  be  statisti¬ 
cally  significant  concerning  quarterly  effects  across 
various  years.  # 
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IV.  CONCLUSIONS 


The  problem  as  defined  was  to  aid  the  Department 
Chairman  in  his  task  of  scheduling  and  budgeting  for  faculty 
teaching  responsibilities  by  providing  either  a  better 
prediction  method  or  adjustments  to  the  present  forecasts, 
from  the  scheduler’s  standpoint,  jumps  across  any  multiple 
of  thirty  students  cause  new  sections  to  be  created  or 
dropped,  and  courses  with  a  demand  of  less  than  five 
students  are  dropped.  To  the  scheduler  or  budget  office, 
these  are  significant  numbers.  The  Department  Chairman, 
however,  cares  little  about  the  actual  number  but  much  about 
the  accuracy  of  the  prediction.  Since  predictions  are  made 
by  course  and  broken  down  within  individual  courses  by 
curriculum,  adjustments  were  found  for  courses  and 
curricula.  Prediction  data  were  the  limiting  quantities; 
predictions  by  curricula  exist  only  as  far  back  as  FY83. 
This  limited  the  scope  of  the  investigation  to  1  and  3/4 
years.  With  this  limited  history,  indications  of  problem  or 
non-problem  areas  are  the  only  adjustment/accuracy  measures 
possible. 

A.  CCMC10 SION S  BY  COURSE 

1  -  Program  Out  put  Utilization  and  Displays 

Indications  of  predictive  accuracy  by  course  were 
obtained  using  the  ’CCURSEP’  and  ’COURSE’  programs  detailed 
in  Appendix  A.  The  'COURSEP'  (standing  for  course  patterns) 
program  was  used  first.  This  program  takes  the  input 
matrix,  in  this  case  the  variable  'OS’  containing  all  of  the 
data  so  far  available,  and  displays  it  grouped  by  course  in 
decreasing  order  of  predictive  error  magnitude  (see  figure 
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9 
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-84.3 

0 

3602 

14 

14 

0 

Figure  4.1  Sample  Output  from  COURSEP  Program 

4.1  for  a  single  course  example).10  This  output  is  quickly 
scanned  for  'interesting*  courses  -  those  with  either 
significant  problems  or  very  small,  infrequent  errors. 
Those  courses  with  problems  are  further  checked  to  determine 
if  there  is  any  pattern  to  the  errors  observed.  Many 
negative  numbers  will  indicate  an  overestimation  problem 
(prediction  greater  than  actual  attending),  conversely  posi¬ 
tive  numbers  indicate  underestimation.  Quarters  may  be 
split  indicating  large  errors  for  some,  small  for  others. 
Curricula  may  be  grouped  together  indicating  prediction 
performance  differences.  in  the  example  shown  (Figure  4.  1)  , 
error  magnitude  is  small  and  signs  vary,  indicating  rela¬ 
tively  minor  deviations  from  the  predicted  values.  Had 
these  errors  been  large,  the  following  observations  could 


10In  this  display  (and  all  others),  the  q 
.  negative  number.  This  is  done  strictly 


uarter  is  shown 
as  a  formatting 
n  in  the  output 


display  deleted  the  six  spaces  'saved'  for  the  sign.  This 
caused  an  irritating  misalignment  in  the  columns  of  the 
display.  The  easy  correction  was  to  ensure  a  negative 
existed  in  all  cases,  thus  the  quarter  coding. 


have  teen  made: 

•  Third  quarter  errors  tend  to  be  higher  than 

first  quarter  errors. 

•  No  curricula  appears  to  do  any  better  or  worse 

than  any  other,  though  374  may  tend  to  over¬ 
estimate  in  their  predictions. 

To  further  detail  predictive  behavior  within  a 
course,  the  'COURSE'  program  is  utilized.  This  program 
takes  the  same  matrix  previously  investigated  (in  this  case 
'OS')  and  also  displays  it  grouped  by  course,  but  with 
various  catagorical  presentations  made.  Output  may  be 
displayed  in  quarterly  or  curricula  subcategories  (through 
answering  program  prompts).  With  the  curricula  option 
selected,  the  output  compares  patterns  of  individual 
curricula  (see  figure  4.2).  Statistics  displayed  include 
the  mean  and  variance  of  the  error  terms  for  individuals 
curricula,  and  predictive,  accuracy  and  no  error  percentages 
also  by  curricula.  To  avoid  counting  data  twice,  summary 
data  entries  (column  2=0)  are  disregarded.  From  scan  of 
the  example  (figure  4.2) ,  the  following  conclusions  can  be 
drawn : 

•  Curriculum  525  provides  the  majority  of  the 

input  for  CS3602,  and  their  predictions 
are  excellent  indicating  no  problem  with 
the  course. 

•  Curriculum  374  may  show  a  pattern  of  over- 

estimation. 

•  Curriculum  686  (with  100%  prediction  errors) 

normally  will  not  provide  inputs  to  this 
course . 
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Figure  4.2  Sample  Output  from  COURSE  Program 
Options  :  Curricula  Grouping  with  Error  Percentages. 

The  curricula  option  of  the  'COURSE'  program  appears 
to  be  the  most  valuable  display  available.  The  information 
shown  is  directly  transf er r ab le  to  the  prediction  data  as 
presented  (refer  back  to  figures  2.1  and  2.2). 

With  the  guarter  option  selected,  the  output 
combines  all  of  the  data  into  four  catagories  depending  on 
the  quarter  in  which  the  course  has  been  taught  (figure 
4.3).  The  same  statistics  are  displayed,  this  time  for  each 
guarter.  Again,  summary  data  entries  are  disregarded  to 
avoid  duplication  of  data.  From  a  scan  of  this  figure,  the 
following  conclusions  can  be  drawn: 
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Figure  4.3  Sample  Output  from  COUBSE  Program 
Options  :  Quarter  Grouping  with  Error  Percentages. 

•  OS3602  has  only  been  taught  during  the  fall 

and  spring  guarters. 

•  Fall  quarter  predictions  tend  to  be  exact. 

•  Spring  quarter  predictions  are  worse,  but  not 

to  such  an  extent  as  to  cause  problems. 


2.  Display  Based  Conclusions  by  Course 

Dsing  the  previously  discussed  displays,  the  courses 
were  separated  into  three  catagories  of  interest:  those 
with  indications  of  significant  prediction  accuracy 


problems,  those  with  relatively  minor  problems  if  any,  and 
those  with  too  little  information  to  draw  conclusions  (see 


TABLE  V 

Course  Results 


Problem  Courses 

2101  2102  2103 

3001  3008  3101 

3102  3103  3104 

3105  3404  3702 


Inadequate  Data 

3003  3005  3302 

3401  3403  4602 


•Good*  Courses 

3002  3004  3006 

3303  3402  3601 

3602  3603  3604 

4601  4701 


table  V).  Predictions  made  for  courses  can  be  checked 
against  this  listing  for  an  indication  of  their  expected 
performance.  Should  more  detailed  information  be  needed. 
Appendix  C  contains  the  results  based  on  summary  data  from 
81-82,  complete  82-83  data,  and  fall,  winter  and  spring 
quarters  from  the  83-84  school  year. 

E.  CONCLUSIONS  BY  CURRICULA 

1  •  Program  Output  Utilization  and  Displays 

Indications  of  predictive  accuracy  by  curricula  were 
obtained  using  the  ’CURRICP’  and  'CURRIC'  programs  detailed 
in  Appendix  A.  The  ’CURRICP*  (standing  for  curricula 
patterns)  program  was  used  first.  This  program  takes  the 
input  matrix,  'OS’  as  before,  and  displays  it  grouped  by 
curricula  in  decreasing  order  of  predictive  error  magnitude 
(see  figure  4.4  for  a  single  curriculum  example).  This 
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CURRICULUM  :  374 
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Figure  4-4  Saaple  Output  from  CORRICP  Program 

output  is  quickly  scanned  for  ’interesting'  courses  -  those 
with  significant  problems  or  small,  infrequent  errors. 
Those  curricula  with  significant  errors  are  further  checked 
for  discernable  patterns  such  as  regularly  occuring  over-  or 
underestimation,  or  quarterly  trends.  In  the  example  shown 
(figure  4.4) ,  problems  appear  due  to  consistent  overestima¬ 
tion.  However,  given  the  magnitude  of  the  errors,  these  may 
be  viewed  as  insignificant.  It  also  appears  that  fall  and 
winter  quarters  tend  to  have  fewer  or  smaller  prediction 
errors. 

To  more  closely  investigate  patterns  shown,  the 
'CURRIC'  program  is  utilized.  This  program  takes  the  same 
input  matrix  and  displays  it  further  grouped  by  quarterly  or 
course  catagories.  With  the  quarter  option  selected,  the 
output  compares  predictions  across  time  (see  figure  4.5). 
From  the  example  shown,  the  following  observations  can  be 
made : 


•  Overestimation  errors  do  increase  across 


consecutive  quarters 


•  Lower  level  courses  tend  to  have  larger  mag¬ 

nitude  errors  than  more  advanced  courses. 

•  OS2103  and  OS3401  might  not  be  taken  by  cur¬ 

riculum  374  students  fall  quarter  (more 
data  required) 
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Figure  4.5  Sample  Output  from  CURRIC  Program 
Options  :  Quarter  Grouping  with  Error  Percentages. 


With  the  coarse  option  selected,  information  can  be 
gleaned  cn  curriculum  374's  prediction  performance  fcr  any 
given  course  (see  figure  4.6).  Again  from  the  example 
shown,  conclusions  can  be  drawn: 

•  OS2101  and  OS3401  are  generally  not  taken  by 

students  from  curriculum  374 

•  Larger  errors  cccur  in  OS2103  and  OS3604  (as 

expected  from  the  course  matrix  -  refer  back 
to  figure  3.2) 

2 .  Display  Based  Conclusions  by  Curriculum 

Using  the  previously  discussed  displays,  the 
curricula  were  separated  into  three  catagories  of  interest: 
those  with  indications  of  significant  prediction  accuracy 
problems,  those  with  relatively  minor  problems  if  any,  and 
those  with  too  little  information  to  draw  conclusions  (see 
table  VI).  Predictions  made  for  courses  can  be  checked 

against  this  listing  for  an  indication  of  their  expected 
performance.  Should  more  detailed  informtion  be  needed. 
Appendix  C  contains  the  results  based  on  summary  data  from 
81-82,  complete  82-83  data,  and  fall,  winter  and  spring 
quarters  from  the  83-84  school  year. 
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Figure  4.6  Sample  Output  from  CURRIC  Program 
Options  :  Course  Grouping  with  Error  Percentages. 
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TABLE  71 

Curricula 

Results 

Pr ctlem 

Curricula 

•Good' 

Curricula 

360 

368 

374 

460  530 

365 

366 

367  377  525 

531 

590 

817 

819  825 

595 

600 

620  813  8  14 

827 

837 

847 

857 

815 

999 

Inadequate  data 

372  373  380  440 

441 

532  535  570  591 
611  683  686  687 

610 

APPENDIX  A 

PROGRAHS  AND  DATA  BASE  OSERS  MANUAL 

This  appendix  contains  the  detailed  information  required 
to  effectively  run  and  maintain  the  programs  and  data  base 
used  for  historical  validation  of  the  Naval  Postgraduate 
School  course  loading  forecasts.  The  purpose  of  this  series 
of  programs  is  to  conveniently  display  historical  course 
attendance  data  versus  predicted  attendance  data.  These 
displays  can  then  be  translated  into  simple  adjustments  to 
be  made  to  the  forecast,  providing  a  more  accurate  figure  to 
be  used  in  the  faculty  teaching  assignment  process. 

A.  ACCESS  PROCEDURES 

The  data  and  manipulative  programs  associated  are  held 
on  the  IBM  3033  mainframe  at  the  Naval  Postgraduate  School 
under  user  number  1502P.  The  are  contained  in  an  APL  work¬ 
space  called  ' OSDA TA ’ .  Assuming  you  have  a  valid  user 
account  on  the  main  NFS  computer  system,  this  workspace  can 
be  accessed  in  three  steps;  linking  to  user  1502P,  accessing 
his  disk,  and  copying  the  workspace  onto  your  own  disk. 
Cnee  you  are  ’logged  on'  to  the  system,  the  command 

CP  link  1 50 2p  195  195  rr 

will  cause  a  prompt  to  appear  asking  for  the  'read'  pass¬ 
word.  Enter  ’ps3000’.  That  password  allows  access  to  user 
1502P  ’  s  disk.  To  get  the  workspace  transferred  to  your  cwn 
disk,  enter 


acc  195  b/a 

copy  osdata  vsaplws  b  =  =  a 


followed  by 


B.  BON  PREPARATION  INSTRUCTIONS 


To  effectively  run  the  programs  available,  data  must  exist 
in  a  six  column  numerical  matrix  format  coded  as  shown: 

Column  Sample 

Number  Entries  Com  ments 

1  83.1  This  column  corresponds  to  the  quarter 
84.4  from  which  the  data  is  gleaned.  '83. 1' 

corresponds  to  the  Fall  quarter  of  the 
1982-1983  school  year  (as  in  standard 
fiscal  year  notation).  Similarly, 

84.4  corresponds  to  the  summer  quarter 
of  the  1983-1984  school  year. 

2  0  This  column  holds  the  curriculum  code. 

374  "O’  means  that  the  entries  to  follow 

are  total  number  of  students  predicted 
and  total  actually  enrolled  in  the 
listed  course  as  compared  to  '374' 
which  means  that  the  figures  to  follow 
apply  only  to  curriculum  374. 

3  2103  This  column  holds  the  OS  course  number 
3602 

4  29  This  column  holds  the  actual  number  of 

0  students  attending  the  course  as  de¬ 
scribed  by  the  previous  columns.  If 
column  two  is  zero,  this  number  is  the 
total  number  of  students  that  attended 
and  should  correspond  to  the  total 
listed  on  the  grade  record  sheet  held 
by  the  registrar.  If  this  number 
itself  is  zero,  the  course  was  not 
taught  during  the  quarter  shown  in 
column  one. 
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5 


0 

15 


0 

5 


This  column  holds  the  predicted  number 
of  students  that  should  attend  the 
course  described.  If  column  two  is 
zero,  this  number  is  the  total  number 
of  students  that  were  predicted  for 
the  course  and  should  correspond  to 
the  totals  listed  on  both  the  summary 
and  individual  course  forecasts.  If 
this  number  itself  is  zero,  either  the 
course  was  predicted  to  be  empty  or 
the  course  was  not  predicted. 

This  column  holds  the  difference 
between  the  actual  number  of  students 
attending  the  course  and  the  number 
predicted  (actual  minus  predicted). 
Negative  numbers  imply  overestima tes, 
positive  numbers  imply  underestimates. 


Prior  to  any  program  runs,  the  goal  of  your  analysis 
must  be  defined.  The  programs  are  generally  divided  into 
two  areas,  course  centered  and  curricula  centered.  In  the 
course  centered  area,  the  'COURSEP'  program  allows  a  general 
viewing  of  the  error  data  for  chosen  courses.  This  program 
displays  the  data  separated  into  courses  ordered  by 
decreasing  magnitude  of  the  error  term  (actual  minus 
predicted;  column  six)  and  is  used  to  check  quickly  the 
prediction  accuracy  by  course.  Then  to  check  for  underlying 
patterns  by  quarter  taught  or  by  individual  curricula,  the 
'COURSE*  program  can  be  used.  Similarly,  in  the  curricula 
centered  area,  the  'CUERICP*  program  allows  a  general 
viewing  of  the  error  data  separated  into  individual 
curricula,  and  the  'CURRIC'  program  can  be  used  to  further 
distill  the  information  available. 


In  summary,  two  steps  must  be  taken  prior  to  starting 
program  runs: 

•  Ensure  the  data  exists  in  the  proper  format 

•  Decide  what  information  is  needed  (define  goals) 

C.  SAMPLE  PROGRAM  RODS 

To  demonstrate  effectively  the  varied  uses  of  the 
programs  available,  a  hypothetical  problem  will  be  created 
and  fully  explored-  Each  program  will  be  discussed  in 

detail  as  it  appears  in  the  problem  solution,  and  both  the 
inputs  required  and  output  generated  will  be  shown. 

1 •  Problem  Statement 

Curriculum  525,  Antisubmarine  Warfare,  and  curric¬ 
ulum  374,  Air-Ocean  Tactical  Environmental  Support,  are 
going  to  combine  parts  of  their  programs  to  create  a  new 
curriculum:  Curriculum  449.5  -  Antiwhale  Warfare.  The 

condensed  course  matrix  for  curriculum  449.5  has  OS2103, 
CS3604,  OS3601,  and  OS3602  in  quarters  2,  3,  4  and  5  respec¬ 
tively.  As  the  Operations  Analysis  Department  Chairman,  you 
know  that  OS2103  and  OS3604  can  handle  the  additional 
students  with  no  problems  due  to  the  present  size  and 
frequency  of  those  course  offerings.  Your  question,  with 
scheduling  coming  up  for  the  next  fiscal  year,  is  whether 
the  additional  12  students  you  expect  per  ’class'  will  be 
enough  to  require  the  offering  of  an  additional  section  for 
OS3601  or  OS3602.  The  figure  of  twelve  students  comes  from 
the  Air-Ccean  curricular  office,  since  curriculum  449.5  will 
le  taught  as  an  adjunct  of  curriculum  374.  These  students 
will  be  entering  in  the  Fall  and  spring  quarters. 
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2  -  Eef inition  of  Goals 

The  first  step  in  the  problem  solution  is  to  define 
the  information  reguired  to  answer  the  guestion.  In  this 
case,  we  need  to  knew  if  an  additional  section  is  reguired 
given  an  influx  of  twelve  additional  students.  This  problem 
defines  three  guestiens: 

•  How  many  students  usually  take  the  course  in 
guestion 

•  How  accurate  are  those  figures 

«  How  good  is  the  estimate  of  twelve  additional 
students 

Since  the  first  items  to  be  considered  involve 
course  data,  the  course  related  programs  will  be  used.  To 
check  later  the  accuracy  of  the  figure  twelve,  a  curricular 
comparison  can  be  made  with  curriculum  374  (since  that 
office  is  the  administrator  of  the  program)  using  the 
curricula  related  programs.  Additionally,  the  data  used 
must  be  the  most  recent  and  complete  information  available. 

2-  Course  Related  Work 

The  first  step  we  will  take  is  to  get  a  look  at  the 
data  of  interest.  The  variable  'OS’  contains  the  informa¬ 
tion  we  want.  The  ’COORSEP*  program  most  easily  shows  this 
data.  Through  proper  inputs,  we  can  call  up  the  historical 
performance  of  OS36Q1  and  OS3602.  The  'COURSEP'  program 
provides  all  the  prompts  necessary  (see  figure  A.1).  The 
program  output  is  shewn  in  figure  A. 2  .  See  figure  A. 3  for 
a  program  listing. 

As  we  can  see  from  figure  A. 2,  the  actual  number  of 
students  that  attend  each  course  appears  to  be  somewhere 
around  15.  The  predictions  are  presently  perfectly  adequate 
since  they  show  a  reguirement  for  only  one  section  in  all 
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INPUT  PROMPT 

COURSE?  OS 

* 

0 

* 

3601  3602 

*  see  COURSEP  listing  for  details 
of  the  prompts 


Figure  A.1  Input  Syntax  :  COURSEP 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


7COURSEP[0]9 


<7  coursep  matrix ; x $  I ;r; z 

*  THIS  FUNCTION,  GIVEN  AH  INRUT  MATRIX  IN  THE  FOLLOWING  FORM  : 


GTR  CUPRIC  CSe  ACTUAL  PRIPICTEP  ( AC T U AL - PR E D I C T E D ) 


WILL  SORT  AND  D I  SPLAY  THE  DATA  PT  CURRICULUM  IN  THE 
FOLLOWING  MANNER  J 


U  DATA  ORDERED  BY  MAGNITUDE  OF  ERROR  TERM 

ft 

„  (ALL  QUARTERS  WILL  BE  DISPLAYED  AS  NEGATIVE  NUMBERS,  THIS  ALLOWS 
ft  FOR  CORRECT  PLACEMENT  OF  THE  OUTPUT  COLUMNS  WITHOUT  RESORTING 
II  TO  COMPLEX  CHARACTER  ARRAY  MANIPULATIONS,) 
ft 

,1  THE  PROGRAM  GOAL  IS  TO  DISPLAY  OBVIOUS  PATTERNS  THAT  MAY  THEN  BE 
ft  LOOKED  AT  MORE  CLOSELY  USING  THE  'CURRIC  OR  'COURSE*  FUNCTIONS, 
ft  COURSES  MAY  BE  SELECTED  INDIVIDUALLY  IF  SO  DESIRED, 
ft 

MATRIX*-MATRIXX  (  f  MATRIX  )  f  ”111111 

t'IF  YOU  WANT  ALL  COURSES  DISFLATED,  ENTER  1* 

f  1  OTHERWISE,  ENTER  Q  T0  SELECT  SOME  SUBSET  OF  THE  COURSES* 

4 JUMPJ x \ Q 

t * THE  FOLLOWING  COURSES  ARE  AVAILABLE  FOR  SELECTION  • • 
tX#-X£*X4*UNIClUE  MATRIXCJ333 

t* ENTER  THE  COURSE  NUMBERS  YOU  WANT,  ENTER  X  TO  GET  THEM  ALL,' 
f* (HOTE  J  YOU  MUST  CHOOSE  MORE  THAN  ONE  COURSE)' 

X4-Q 

4JUMP2 

JUMP  1  ♦  Y  4-  fXfX[  *:<«.  UNIQUE  MATRIX^  »3]3 

JUMF2  J  I  «-  1 
CRSLOOP't'  ' 

40X  I  X  >  Y 

(  r  •  COURSE  J  '  )  »fX[I] 

Z«-(XCI3=MATRXXCr33)/Cn  MATRIX 
t-z  ORDER  £ 


INPUT  PROMPT 

COURSE  OS 

* 

0 

* 

3601  3602 

* 

0 

* 

n 

*  see  COURSE  listing  for  details 
of  the  prompts 


Figure  A. 4  Input  Syntax 


_ i 

COURSE 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


vcoursecQ] v 

v  course  mat* ix ; x ;  i ;  y  ;  2 ;  mc  ;  ve ;  u ;  ;<2 >  T2 i mo  ;  vo ;  w ;  qtr  ; no ; nc  ;  ns  ; me ;  ind ; , 

Cl]  a 

C2]  a  this  function,  oiven  on  inrut  matrix  in  the  pollomihs  form  ; 

C33  * 

C4]  a  qtr  curric  cse  actual  predicted  (actual-predicted) 

C31  a 

C6]  a  will,  sort  ond  display  the  data  »y  curriculum  in  the 
C7]  A  FOLLOW I NO  MANNER  I 

C8]  a 

C9]  A  COURSE 

CIO]  A  QUARTER  1  DATA  CURRICULUM  l  data 

Cli:  a  quarter  2  data  or  curriculum  2  data 

C12]  A  QUARTER  3  DATA  , 

C13]  A  QUARTER  4  DATA  CURRICULUM  N  DATA 

C14]  A  MATRIX  OR  GROUP  MEANS,  VARIANCES  OND  N 

CIS]  A  OVERALL.  MEAN,  VARIANCE  OND  M 

C  1  6]  A 

C17]  A  A  MATRIX  OP  ACCURACY,  PREDICTIVE  AND  NO  ERROR  PERCENTAGES  IS 
C18]  A  AVAILABLE  IP  DESIRED, 

C  19]  A 

C20]  A  (ALL  QUARTERS  MILL  SE  DISPLATED  AS  NEGATIVE  NUMDERS ,  THIS  ALLOWS 
C21]  A  FOR  CORRECT  PLACEMENT  OP  THE  OUTPUT  COLUMNS  WITHOUT  RESORT I HG 
C22]  A  TO  COMPLEX  CHARACTER  ARRAY  MANIPULATIONS,) 

C23]  A 

C24]  A  lltlllll  NOTE  ;  THE  MEANS  AND  VARIANCES  COMPUTED  APPLY  AAAAAAAA 

. . .  *  A  A  ONLY  TO  THE  INBISSIBUSli  CIJRRIEULa  iiilj  AAAAAAAA 

C26]  A  AAAAAAAA  ANY  SUMMARY  DATA  (-(COLUMN  2  ENTRY  w  0)  AAAAAAAA 

C27]  A  AAAAAAAA  IS  COMPRESSED  OUT  OP  THE  COMPUTATION,  AAAAAAAA 

C28]  A  AAAAAAAA  THIS  CAN  AS  CHANGED  ST  REPLACING  THE  4  AAAAAAAA 

C29]  A  AAAAAAAA  SIGN  IN  THE  LOOPS  (WHERE  2  OR  W  C»2]  *•  AAAAAAAA 

C30]  A  AAAAAAAA  COMPRESSING  COLUMN  £  OR  WHERE  X2  IS  AAAAAAAA 

C31]  A  AAAAAAAA  COMPRESSING  THE  OUTPUT)  WITH  's',  THISaAAAAAAA 

C32]  A  AAAAAAAA  FUNCTION  WILL  THEN  DISPLAY  ONLY  THE  AAAAAAAA 

C333  A  AAAAAAAA  SUMMARY  DATA.  AAAAAAAA 

C34]  a 

C33]  A  COURSES  MAY  BE  SELECTED  INDIVIDUALLY  IP  SO  DESIRED, 

C34]  a 

C37]  matrix«.matrix«(amatrix)p  *111111 

C38]  f 1  IF  YOU  WANT  all’  COURSES  DISPLAYED,  ENTER  J  • 

C39]  r'OTHERAISI,  ENTER  0  T0  SELECT  SOME  SUBSET  OP  THE  COURSES' 

C40]  -»JUMPlx,(] 

C41]  T' THE  FOLLOWING  COURSES  ARE  AVAILABLE  FOR  SELECTION  •• 

C42]  YMLXC4XLUHIQUE  MATRIXES}] 

C43]  » ' ENTER  THE  COURSE  NUMBERS  YOU  WANT,  ENTER  X  TO  GET  THEM  ALL.' 

C44]  r ' (MOTE  :  YOU  MUST  CHOOSE  MORE  THAN  ONE  COURSE)' 

C43]  YLfXLQ 
C46]  yuump; 

C47]  JUMPi  ;YLf  xpxc*<<puHiauE  matrix(;j333 
C48]  jump]!  Ul 

C49]  » ■ the  data  may  be  displayed  (to  looa  at  patterns)  in  groupings  by ■ 

C30]  f 'CITHER  QUARTER  OR  CURRIC.  FOR  QUARTER  GROUPINGS,  ENTER  1  (  FOR' 

C31]  r' CURRIC  GROUPINGS,  ENTER  0‘ 

C32]  vlQ 

C53]  »' ACCURACY  ERROR,  PREDICTIVE  ERROR,  AND  NO  ERROR  PERCENTAGES  ARE' 

C34]  T' AVAILABLE,  ENTER  Y  IP  DESIRED,  N  OTHERWISE' 

CSS]  inolq 
C56]  crsloop; • 

CS7]  -»0n  i  >  i 

CS8]  (t'COurse  ;  1 ) , pXC I ] 

C39]  Cp(xci-]=MATRixn33)/c13  matrix 


Fiaure  A.  6  «coiirse»  PMura# 


REPRODUCED  AT  GOVERNMENT  EXPENSE 


C603  *»cvMCAH<=cj22^o>/=c;ia 
C&13  vCYvaRiA(Zc;23P0>/zC.63 
C&23  NC*-f  (ZCJ23P0)/2Ci&3 
C433  'JH 
C  643  -.«c;<tx,v^i 

C&53  OTR*-10*(-2Cil3)-L-zCil3 

C663  tout 

C673  next  j  qtr .23 

C683  out ; T2ypx2yx2C*x2yunioue  aiR] 

C693  NSxRExPtfxamoxyotU) 

C703  Otrloop  ;  -*EHD*  \  u  >  Y2 
C71  3  -•■<  <:<2C'<:  =  aTR)/ci2  3)  order  4 

C723  MaYHa,MEAM<wc;23*0)/«C;63 

C733  vo*-WQ,vaRia(wc;23^0)/wCi63 
:743  No*.Na,NE«-p(i»c;23pio)/“Ci6a 
C753  4JUMP3X  \  inDss  1  h  * 

C763  'Eppe,  (Y/<«C»23X0>/<*<Ci43=0>vMCJ53»0>^*e 

C773  »*«-<»«.  (+/<“C  »23»!0)/<“C}4 3»»0  >/><“£  i  33^0 

C783  NS«-HSt  (y/(wcJ23P0>/<«C»43P0>a(«*C;S3P0>''MC»A3  =  0>yme 

C793  JUMP3;,* 

C803  y 1  • 

C81 3  JpJ»l 
C823  pOTrloop 
C833  exoiinti 

C843  T '  GROUP  MEAN  VARIANCE  N  (OP  ERROR  TERM)' 

C833  7  1  t(X2^0)/C13( (*X2»C0.S3  mO),T23  vq>,C23  no 

C863  T'  ' 

C873  ( r 1 OVERALL :  •),  &  1  tXC,VC,NC 

C883  y'  ' 

C893  YJUMP4X  (  IND  =  •  N  • 

C903  Y*  ERROR  PERCENTAGES' 

C913  y'  OROUP  Ppg  92£  hone  B' 

C923  7  2  y<X2PO)/C13( ( (OX2.C0.S3  pe>,C23  »e>.C23  «*>,C23 

C933  y'  • 

C  94  3  JUMP4 2  y 1  ' 

C953  -tCRSLOOP 

v 


[•111 


INPUT 


FROMPT 


COURSE  OS 

* 

0 

* 

3601  3602 

* 

1 

* 

n 


*  see  COURSE  listing  for  details 
of  the  prompts 


Figure  A. 7  Input  Syntax 


COURSE 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


COURSE  ♦ 

3601 

”82.2 

0 

3601 

14 

8 

6 

"83.2 

0 

3601 

14 

16 

—  o 

4m 

"84.2 

365 

3601 

n 

0 

n 

"84.2 

525 

3601 

20 

no 

“•  n 

"83.2 

374 

3601 

4 

5 

"i 

"83.2 

525 

3601 

10 

ii 

"i 

"84.2 

0 

3601 

25 

26 

"i 

"84.2 

374 

3601 

3 

4 

"i 

"83.4 

0 

3601 

14 

23 

"9 

"83.4 

365 

3601 

3 

11 

"8 

"82.4 

0 

3601 

14 

21 

“7 

"83.4 

374 

3601 

7 

5 

"3 

"83.4 

525 

3601 

9 

'  7 

O 

GROUP 

MEAN 

VARIANCE  N  (OF 

ERROR 

term  > 

2.0 

".6 

2.3 

5.0 

4.0 

"3.0 

25.0 

3.0 

overall; 

"1.5 

10.0 

8.0 

COURSE  ♦ 

3602 

"82.1 

0 

3602 

7 

n 

"4 

"83.1 

0 

3602 

7 

7 

0 

"83.1 

374 

3602 

1 

1 

0 

"83.1 

525 

3602 

6 

6 

0 

"84.1 

0 

3602 

12 

12 

0 

"84.1 

374 

3602 

3 

3 

0 

"84.1 

525 

3602 

9 

9 

0 

"83.3 

374 

3602 

3 

5 

—  n 

4m 

"84.3 

374 

3602 

2 

4 

—  n 

"8*  .3 

525 

3602 

12 

10 

n 

"82.3 

0 

3602 

15 

14 

i 

"83.3 

0 

3602 

15 

16 

"i 

"83.3 

686 

3602 

1 

0 

i 

"83.3 

525 

3602 

11 

11 

0 

"84.3 

0 

3602 

14 

14 

0 

GROUP 

MEAN 

VARIANCE  N  (OF 

ERROR 

TERM  ) 

1.0 

.0 

.0 

4.0 

3.0 

".2 

3.2 

5.0 

OVERALL ; 

".1 

1.6 

9.0 

Figure  A. 8  Output  from  Input  Sequence  shown  in 
Figure  A. 7  :  COURSE  Program. 
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4 .  Curricula  Related  Work 

In  looking  at  the  curricula  information,  we  want  to 
find  out  just  how  good  the  figure  ’twelve’  is  for  the 
expected  input  from  the  Antiwhale  Warfare  curriculum.  Since 
curriculum  374  will  te  administering  the  program,  we  will 
use  the  data  from  374  as  an  approximation  of  curriculum 
449. 5's  performance.  Again,  the  variable  'OS’  contains  the 
information  we  want.  The  ’CCJREICP'  program  most  easily 
shows  this  data.  Through  proper  input,  we  can  call  up  the 
historical  performance  of  curriculum  374  .  The  'CURF.IC?' 
program  provides  all  the  prompts  necessary  (see  figure  A. 9) . 
Since  we  are  only  interested  in  one  curriculum  but  two  are 
reguired  for  proper  program  execution,  a  dummy  curriculum 
(333)  is  requested.  The  program  output  is  shown  in  figure 
A.  10  See  figure  A. 1  1  for  a  program  listing. 

As  we  can  see  from  figure  A.  10,  curriculum  374 
routinely  appears  to  overestimate  the  number  of  students 
expected  for  any  given  course.  To  show  more  adequately 
patterns  of  concern  in  the  data  (such  as  performance  for 
OS3601  and  OS3602) ,  we  can  look  at  the  'CURRIC'  program 
sorted  by  course.  This  will  break  out  the  predictions  to 
tetter  show  how  well  curriculum  374  performs  for  individual 
courses.  Figure  A.  12  shows  the  inputs,  figure  A.  13  the 
outputs,  and  figure  A. 14  the  program  itself. 

As  we  can  now  see  from  the  course  data  shown  in 
figure  A.  13,  curriculum  374  generally  overstates  the 
expected  number  of  students  attending  courses  OS3601  and,  to 
a  lesser  extent,  053602.  We  can't  observe  any  possible 
patterns  by  quarter  due  to  the  limited  number  of  observa¬ 
tions.  A  brief  look  at  the  error  percentage  table  shows  no 
predictive  errors  for  either  course  (they're  taught  or  not 
taught  when  they  say  so  -  no  'zeros'  in  the  actual  or 


predicted  columns),  and  an  improvement  in  accuracy  from  3601 
to  36C2  (03  error  free  to  503  error  free  respectively) . 

In  summary,  through  use  of  the  curricula  related 
programs  we  have  found  that  a  slight  overestimation  of 
students  expected  is  usual  for  these  courses.  This  trend  may 
or  may  not  hold  for  increased  class  sizes  (of,  say,  twelve 
students  vice  three  or  four).  Therefor,  combined  with  the 
information  we  previously  obtained  from  the  course  rfelated 
programs,  we  conclude  that  new  sections  will  probably  not  be 
required. 


INPUT  PBOMFl 


CUERICP  OS 

* 

0 

* 

374  333 


*  see  COURSEP  listing  for  details 
of  the  prompts 


Figure  A. 9  Input  Syntax  :  CURRICP 


REPRODUCED  AT  GOVERNMENT  EXPENSE. 


Figure  A. 10  Output  from  Input  Sequen 
Figure  A. 9  :  CDEBIC?  Program 


uence  shown  in 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


7CURR I 

<?  curricp  MATRixfx; i;t;: 

cm  a 

[23  A  THIS  FUNCTION,  GIVEN  AN  INRUT  MATRIX  IN  THE  FOLLOWING  FORM  J 

C33  a 

[43  A  QTR  CURRIC  CSE  ACTUAL  PREDICTED  ( ACTUAL— PREDICTED  > 

C33  A 

Z61  A  WILL  SORT  AND  DISPLAY  THE  DATA  BY  CURRICULUM  IN  THE 
C73  A  FOLLOWING  MANNER  J 

C83  a 

[93  ft  CURRICULUM 

C103  A 

llll  ft  DATA  ORDERED  BY  MAGNITUDE  OF  ERROR  TERM 

C123  A 

C133  A  (ALL  OUARTERS  WILL  BE  DISPLAYED  AS  NEGATIVE  NUMBERS,  THIS  ALLOWS 
[143  A  FOR  CORRECT  PLACEMENT  OF  THE  OUTPUT  COLUMNS  WITHOUT  RESORTING 
C153  A  TO  COMPLEX  CHARACTER  ARRAY  MANIPULATIONS,) 

C163  A 

C173  A  THE  GOAL  OF  THIS  PROGRAM  IS  TO  DISPLAY  OBVIOUS  PATTERNS  THAT 
C183  A  MAY  THEN  BE  LOOKED  AT  MORE  CLOSELY  USING  THE  'COURSE*  OR  'CURRIC' 
[193  A  FUNCTIONS,  CURRICULA  MAY  BE  SELECTED  INDIVIDUALLY  IF  SO  DESIRED, 
C203  A 

[213  MATRIX«-MATRI  X*  (  f  MATRIX  )  f  “  1  1  1  1  1  1 

[223  Y ' IF  YOU  WANT  ALL  CURRICULA  DISPLAYED,  ENTER  1» 

[233  ‘  OTHERWISE ,  ENTER  0  TO  SELECT  SOME  SUBSET  OF  THE  CURRICULA' 

C24  3  -+UUMP1  x  \  Q 

[253  f'TMC  FOLLOWING  CURRICULA  ARE  AVAILABLE  FOR  SELECTION  •• 

[263  matrix[?233 

C273  y'ENTCR  THE  CURRIC  NUMBERS  YOU  WANT,  ENTER  X  TO  GET  THEM  ALL, 1 
[283  r' (WOTE  :  you  must  enter  more  than  one  curriculum) 1 
C29  3  r«-fx*.Q 
[303  YUUMP2 

[313  UUMPl  :  Y«-f  X*X[  iXfUNIQUE  matrxx[;233 
[323  JUMP2SIP1 
[333  CRCLOOP;r‘  • 

C343  -»0x\I>y 

C  35  3  ( T  '  CURRICULUM  :  *  )  f  Y*c  X  3 

C36  3  2*.(XCl3=«ttTRixc  ;  23) /Cl 3  matrix 

C373  ORDER  ^ 

C383  Ipi+1 
C393  y 1  ' 

C  40  3  y '  1 

C413  ycrcloop 
v 


pi  mtro  a. 11  •CURRICP*  Proaram 


INFUT  IR0MP1 

CUPRIC  OS 

* 

0 

* 

31H  333 

* 

3 

* 

y 

*  see  COURSE  listing  for  details 
of  the  prompts 


Figure  A.  12  Input  Syntax  :  CORRIC 


REPRODUCED  AT  GOVERNMENT  EXPENSc 


CURRICULUM  :  374 

'33.4  374  2101 


0  6  ~4 


'33.4 

374 

2103 

'33.2 

3/4 

2103 

'34.1 

374 

2103 

'34 . 2 

374 

2103 

'3  4. 1 

374 

3401 

'33.4 

374 

3601 

'33.2 

374 

3601 

'34 . 2 

374 

3601 

"33.3 

374 

3602 

'34.3 

374 

3602 

"33.1 

374 

3602 

'34.1 

374 

3602 

'33.4 

374 

3603 

'84.1 

374 

3603 

"83.3 

374 

3604 

'84.1 

374 

3604 

'83.2 

374 

3604 

'B4.3 

374 

3604 

'83.1 

374 

3604 

GROUP 

MEAN 

VARIANCE  M 

2101.0 

'6.0 

1.0 

1.0 

2103.0 

"2.3 

2.3 

4.0 

3401.0 

'1.0 

1.0 

1.0 

3601.0 

'1.7 

1.3 

3.0 

3602.0 

'1.0 

1.3 

4.0 

3603.0 

'1.0 

2.0 

2.0 

3604.0 

'1.4 

1.3 

5.0 

OVERALL  ; 

'1 . 7 

2.3 

20.0 

ERROR  PERCENTAGES 

GROyp 

r;S 

ess 

tj  g  n  g 

2101.00 

1 . 00 

.00 

.00 

2103.00 

.25 

.  75 

.‘00 

3401 . 00 

l  .00 

.00 

.00 

3601.00 

.00 

1.00 

.00 

3602.00 

.  00 

.50 

,  50 

3603 . 00 

.  00 

.  30 

.50 

3604 . 00 

.20 

,  60 

.20 

CURRICULUM  :  333 

GROUP 

MEAN 

VARIANCE  N 

OVERALL 

:  1.0 

.  0 

.  0 

ERROR 

PERCENTAGES 

GROUP 

ees 

ACC 

NOME 

3  4-4 

1  4  *3 

0  1  -1 

2  3  '1 

0  1  -1 

2  5-3 

<  5~1 

3  4  “1 

3  5  -2 

2  4  '2 

1  1  0 

3  3  0 


4  4  -3 

3  1  0 


4  4  0 

(OR  ERROR  TERM) 


N 

1.00 
4.00 
1 . 00 
3.00 
4 . 00 
2.00 
3.00 


(OR  ERROR  TERM) 


A. 13  Output  from  Input  Sequence  shown  in 

Piaure  A.  12  :  CtlHRIC  Proaram. 
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» 


UCURRICCO}? 


C 1  3 

C23 

C3] 

[4] 

C5] 

C6] 

C7] 

C8] 

C9] 

CIO] 

Cll] 

C 1 2] 

C 1 3] 

C14] 

C 1 5] 

C16] 

C  1 7] 

CIS] 

C  19  ] 

C20] 

C21] 

C22] 

C23] 

C24] 

C2S] 

C26] 

C27] 

C28] 

C  29] 

C30] 

C31] 

C32] 

C33] 

C34] 

[35  ] 

C36] 

C37] 

C  38  ] 

C39] 

C  40  ] 

C  4  1  ] 

C  42] 

C  43] 

C  44  ] 

C  45  ] 

C  4  A  ] 

C  47] 

C  48  ] 

C49] 

C50] 

C51  ] 

C52] 

C531 

C54  ] 

C55] 

C  56  ] 

C57] 

C  58  ] 

C59  ] 


?  cudric  matrix;») : ;z {mc ;vc; j;x2) f2iMO|voiH;i>;Hs;NE ;ind;ac;pe ; v;nc 

ft  THIS  FUNCTION,  GIVEN  OH  IWFUT  MATRIX  IN  THE  FOLLOWING  FORM  • 

ft  QTR  CURRIC  CSC  ACTUAL  PREDICTED  (  (ACTUAL  —  PREDICTED ) 

ft  WILL  SORT  AND  DISPLAT  THE  DATA  BY  CURRICULUM  IH  THE 
ft  FOLLOWING  MANNER  • 

«  CURRICULUM 

ft  •  QUARTER  1  DATA  COURSE  1  DATA 

ft  QUARTER  2  DATA  OR  COURSE  2  DATA 

ft  QUARTER  3  DATA 

*  QUARTER  4  DATA  COURSE  M  DATA 

H  MATRIX  OF  GROUP  MEANS,  VARIANCES  AND  N 

ft  OVERALL  MEAN,  VARIANCE  AND  N 

ft  A  MATRIX  OF  ACCURACY,  PREDICTIVE  AND  NO  ERROR  PERCENTAGES  IS 
ft  AVAILABLE  IF  DESIRED, 

ft  (ALL  QUARTERS  WILL  BE  DISPLAYED  AS  NEGATIVE  NUMBERS,  THIS  ALLOWS 
ft  FOR  CORRECT  PLACEMENT  OF  THE  OUTPUT  COLUMNS  WITHOUT  RESORTING 
ft  TO  COMPLEX  CHARACTER  ARRAY  MANIPULATIONS,) 

ft  CURRICULA  MAY  BE  SELECTED  INDIVIDUALLY  IF  SO  DESIRED. 

MATRIXftMATRIXx (fMATRIX)f  "111111 
, 1  IF  YOU  WANT  ALL  CURRICULA  DISPLAYED,  ENTER  1' 

ft 'OTHERWISE,  ENTER  0  TO  SELECT  SOME  SUBSET  OF  THE  CURRICULA' 
ftJUMP}X |Q 

ft ' THE  FOLLOWING  CURRICULA  ARE  AVAILABLE  FOR  SELECTION  •• 

»xt-:<[4x*-UHiQuc  matri*c;2]] 

ft'EHTER  THE  CURRIC  NUMBERS  YOU  WANT,  ENTER  X  TO  GET  THEM  ALL,' 
ft' (NOTE  •  YOU  MUST  ENTER  MORE  THAN  ONE  CURRICULUM)' 

YftfXftQ 

-»JUMP2 

JUMP1 • Yftf X«.X[ BXftUNIQUE  MATRIX[)2]] 

JUMP 2  ,  It-l 

ft ' THE  DATA  MAY  BE  DISPLAYED  (TO  LOOK  AT  PATTERNS)  IN  GROUPINGS  BY' 
ft' EITHER  QUARTER  OR  COURSE.  FOR  QUARTER  GROUPINGS,  ENTER  1  (  FOR' 

ft ' COURSE  GROUPINGS,  ENTER  3,' 

VLQ 

ft 'ACCURACY  ERROR,  PREDICTIVE  ERROR  AND  HO  ERROR  PERCENTAGE  MATRIX?' 
ft'EHTER  V  OR  N* 

INDftfJ 

CRCLOOP ( ft '  ' 

ftOX  I  I  )  Y 

( ft ' CURRICULUM  •  ’),ftX[I] 

Z«-(XCI]=MATRIXt;2])/tl]  MATRIX 

MCftMEAM  ZC56] 

VCftVAPIA  ZC.6] 

NCftf 2C i6] 

J«-l 

ft  N  E  X  T  X  ) V  =  3 

aPlO»(-zc>v])-t-^C)v] 

ftOUT 

NEXT • AftZ[ ( V] 

out; y2pf«2pN2C*!<2«-unique  a] 

NSft AEftPEftNOftMQft VO ft  I  0 
QTRLOOP • ft END X  ) J  >  Y2 


HO 


ft 


» 


» 


Figure  k.  14 


'CURRIC*  Program 
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REPRODUCED  AT  GOVERNMENT  EXPENSE 


««-(  <*2C  J3=A)/C13  2)  0RDeR  <6 

M  G  *-  M  O  f  MEAN  WCJ63 
VOfV QjVflRIfi  «C»63 
HOfMO  ,  MB4-f  MC 
-+JUMP3X  \  IN  Os  '  N  1 

PE4-PC,  <+/<  i  4  3  =  0)  vwc  ;  53=0  )+hb 
flEfflE, ( ♦/ ( «£  r43*0)A(wt  »  5  3  *0 ) aWC  ?  6  3  *0 ) 
us*. ns  f  (+/<wc  5  4  3*^0)  a(wc» 53^0)  a«C  JA3=*0)^-ME 

JUMP3JtW 

p '  ' 

*OTRLOOP 
END  ♦  I  *-14-1 

GROUP  MEAN  VARIANCE  M  (OF  ERROR  TERM)1 

7  1  f ( <*X2|C0.53  mq>’C23  vo),c23  ho 

t 1  ' 

(  4 1  overall  ♦  •)*  6  1  T*CfVCfNC 


4JUHP4X \ X  NDs '  N ' 
t«  ERROR  PERCENTAGES' 

r  '  GROUP  PRg  ACC  NO Mg  N • 

7  2  p< ( (HX2f CO. 53  *e>,C23  AE>»C23  ns),C23  ho 

r '  # 

JUMP  4 ♦  f  1  • 

4CRCLOOP 


D.  DATA  COLLECTION  AND  MAINTENANCE 

To  provide  the  matrix  of  historical  data  to  be  displayed 
and  analyzed,  two  sources  are  used.  The  prediction  data 
comes  from  the  course  loading  forecast  delivered  to  the 
department  chairmen  (see  figure  A.  15).  The  easiest  way  to 
compile  that  data  is  to  obtain  a  copy  of  the  individual 
course  predictions  and,  on  the  forecast  itself,  change  the 
letter  codes  found  in  the  "S/D/T"  column  into  curricula 
numbers  (using  the  ceding  list  contained  in  appendix  B)  with 
associated  totals.  For  example,  the  totals  you  would  write 
in  from  figure  A. 15  for  OS3602  are  525:  9,  374:  3,  and  0 

(total)  :  12.  The  ether  document  required  is  the  course 
grade  sheet  held  by  the  registrar  (see  figure  A.  16) .  This 
information  is  subject  to  the  privacy  act  and  requires  a 
written  reguest  for  permission  from  the  department  chairman 
involved  to  the  office  of  the  registrar  for  access  tc  the 
files.  The  records  themselves  are  kept  according  to 

academic  year.  This  notation  is  confusing;  to  prevent  prob¬ 
lems,  ask  for  something  like  the  ’83-84  school  year'  data. 
These  records  are  not  allowed  to  leave  the  office,  and  at 
last  check  were  not  available  on  the  computer.  The  data 
must  be  recorded  by  hand.  A  convenient  format  is: 

Quarter  Course  Section  Total#  #A ttending/C urricula 

Once  the  above  listed  data  have  been  collected  and  are 
in  front  of  you  and  a  computer  terminal,  a  CMS  file  as  shown 
in  figure  A. 17  can  be  created  by  combining  the  information. 
The  columns  correspond  to  the  quarter,  curriculum,  course, 
actual  and  predicted  number  of  students.  The  CMS  file  thus 
created  can  be  transferred  to  APL  and  named,  for  instance, 
CSNEXT  through  the  following  command  sequence: 

(get  into  APL) 


) LOAD  990  CMSIO 


) KSID  HOLD 
OSNEXT  <=  CMSREAD 

(follow  prompts  to  read  your  CMS  file) 

) ERASE  CMSIO 
)  SAVE 

NOTES  :  1)  The  APL  variable  'OSNEXT1  now  exists 

on  your  disk  in  the  APL  workspace  ’HOLD' 

2)  The  *<='  character  shown  corresponds  to 
the  left-pointing  assignment  arrow  in  APL 

To  add  the  (Actual  minus  Predicted)  term  to  the  matrix, 
enter : 

CSNEXT  <=  OSNEXT,[  2  ]  OSNEXT[;4]  -  OSNEXT[;5] 

To  add  this  matrix  to  the  variable  OS  (assuming  that  OS  is 
in  the  workspace)  ,  enter: 

CS  <=  OS,[ 1  ]  OS  NEXT 

The  entire  updating  procedure  should  take  about  one  full 
workday  for  each  year  of  data. 
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Figure  A. 15  Individual  Course  Prediction  Format 


OFFICIAL  GRADE  ROSTER  PAGE  1 

NAVAI  POSTGRADUATE  SCHOOL 
MONTEREY, CALIFORNIA  93943  DATE  13  APR 


COURSE  NO.  OS3602  SEGEMENT  00  -  QUARTER  1  AY  83-84 

COMBAT  MODELS  PROF.  SHUDDE,  REX  H. 


3  LECTURE  HOURS  1  LABORATORY  HOURS 


1. 

NAME 

xxxxxx. 

xxxx 

x 

RANK 

LT 

CORPS/ 

COUNTRY 

USN 

CURR 

#374 

MARK 

A 

2. 

xxxxxx. 

xxxx 

X 

LCD  R 

US  N 

#374 

B 

3. 

xxxxxx. 

xxxx 

X 

LCDR 

USN 

#525 

B 

4. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

5. 

xxxxxx. 

xxxx 

X 

LCDR 

USN 

#525 

A- 

6. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

A 

7. 

xxxxxx. 

xxxx 

X 

LCDR 

USN 

#374 

A- 

8. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

A 

9. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

3 

10. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

11. 

xxxxxx. 

xxxx 

X 

LCDR 

USN 

#525 

B 

12. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

FCRM  4 


THIS  IS  THE  OFFICIAL  GRADE  ROSTER  TO  BE  FILED  IN  THE 
ACAEEMIC  DEPARTMENT 


E.  OTHER  AVAILABLE  PROGRAMS  AND  DATA 

A  complete  listing  of  the  functions  available  in  the 
•OSDATA’  workspace  is  shown  in  table  VII  -  The  COURSEP, 
COURSE,  CURRICP  and  CURRIC  programs  have  been  previously 
demonstrated.  The  orly  non-trivial  function  remaining  to  be 
shown  is  the  'WILCOXSE’  program  (see  figure  A. 18).  As  shown 
in  the  figure,  this  program  is  based  on  the  Wilcoxian  Signed 
Rank  Test**  detailed  in  [Ref.  5  pp  280-288]. 

A  self-descriptive  listing  of  the  variables  contained  in 
the  workspace  is  shown  in  table  VIII  .  The  latest  data  pres¬ 
ently  available  is  the  spring  of  1984  (academic  year  83-84) 
figures. 


**For  a  complete  description  of  the  various  hypotheses 
that  may  be  explored,  see  [Ref.  5  pp  280-288].  In  the  body 
of  the  thesis,  the  magnitudes  of  the  forecasting  errors  (y) 
were  tested  against  the  magnitudes  of  prediction  errors 
derived  from  course  matrices  (x) .  The  hypothesis  tested 
(with  rejection  the  desired  result)  was  that  the  ’x*  values 
tend  to  be  egual  to  or  larger  than  the  *  y'  values.  This  was 
one  of  six  possible  interpretations  of  the  null  hypothesis. 
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TABLE  711 

Functions  Available  | 

1 

Function 

******** 

Syntax 

*  ******  * 

Description 

**********************  ******** 

CCURSEF 

COUHSEP  mtx 

{previously  shown) 

CCOSSE 

COURSE  mtx 

(previously  shown) 

CORRICP 

CURRICP  mtx 

(previously  shown) 

CUERIC 

CURRIC  mtx 

(previously  shown) 

WIICCXSR 

y  W ILCOXSR 

x  Shows  the  Wilcoxian  Signed 

Rank  Test  results  (see 
description  in  text) 

ORDER 

A  ORDER  E 

Orders  rows  of  matrix  A 
according  to  decreasing  mag¬ 
nitude  of  column  B,  returning 
the  ordered  matrix 

UNIQUE 

UNIQUE  A 

Returns  the  unique  elements 
of  the  vector  or  matrix  A  in 
ascending  order 

MEAN 

MEAN  X 

Returns  the  arithmetic  mean 
of  vector  X 

V  ARIA 

VARIA  X 

Returns  the  unbiased  estimate 
of  the  variance  of  vector  X 

76 


REPRODUCED  AT  GOVERNMENT  EXPENSE 


5KILCOX5R CD3? 

<7  r  WILCOX  SR  ;,'>0}T;R 


cn 

C23 

THIS  FUNCTION  PROVIDES  THE  MILCOXIAN  SIGNED  RANK 

TEST 

CG3 

STATISTIC  (FROM  CONOVER  PG,  280)*  INPUTS  A  AND  T 

ARE 

C4] 

DATA  VECTORS  OF  gQUAl,  (aitlSiy* 

C51 

Z61 

D4-(  O^0)/Df  f-X 

C73 

Rf.  (  RANK  1  D  )  x0*|  D 

C  8] 

T*< +/">+< +/« *2 >  *0.5 

C9] 

(t’TEST  STATISTIC  a  ')ltT 

V 

Figure  A. 18  Hi lcoxiaa  Signed  Sank  Test  Prograa 
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TABLE  VIII 
Variables  Available 


V  ABIAELE  SHAPE  DESCRIPTION 

********  *******  ************ 


0  S82  69x6 

CS83  255x6 

OSTRK83  140x8 

OS84  228x6 

CS  552x6 

OA83  113x9 


A I  (Academic  year}  81-82  data; 
includes  summary  information  only 
(no  data  for  individual  curricula) 

AY  82-83  data;  complete  listing 

AY  82-83  non-entry  course  data.  The 
additional  columns  contained  in  this 
matrix  (columns  6  and  8)  correspond 
tc  an  'adjusted  prediction'  and  an 
'adjusted  error'  term,  derived  from 
the  course  matrices.  This  array  was 
used  to  check  the  feasability  of 
making  adjustments  based  solely  on 
the  'track'  a  student  takes  using  the 
Kilcoxian  Signed  Rank  Test  previously 
mentioned. 


AY  83-84  data;  only  gtrs  1,2  and  3 

Combination  of  all  of  above 
(without  repetition) 


This  matrix  is  set  up  in  a  different 
format  from  those  previously  dis¬ 
cussed.  The  nine  columns  represent 
Quarter,  OA  course  number.  Total 
attending,  USN  attending,  USA 
attending,  USHC  attending.  Others 
attending.  Total  predicted,  and  Total 
attending  minus  total  predicted, 
respectively.  The  original  intent, 
dropped  due  to  time  constraints,  was 
tc  use  these  figures  to  aid  in  OA 
elective  scheduling  by  looking  at 
expected  number  of  upcoming  free 
electives  by  service  through  use  of 
the  OA  course  matrix. 


APPENDIX  B 
SCHEDULING  CODES 


This  appendix  contains  an  alphabetical  listing  of  the 
specialty  codes  used  in  the  scheduling  process  as  of  August 
1984.  This  listing  cnly  uses  the  first  two  letters  of  any 
individual  coding  because  the  remaining  letters  specify 
either  the  degree  awarded  or  the  degree  and  month  of  input, 
neither  cf  which  change  the  curriculum  associated.  Using 
these  codes,  numbers  of  students  predicted  by  any  given 
curriculum  for  any  given  course  can  be  taken  from  the  indi¬ 
vidual  course  forecast  promulgated  by  the  programming 

office.  Code  0301  at  extension  2773  at  the  Naval 

Postgraduate  School  can  be  contacted  for  updated 
information. 

Specialty 
Curricula  Degree 

Numbers  Codes  Frogram 

it***************************************************** 

610  AC —  Aero  Engr,  incl  Engr  Sci 

AT — 

611  AX —  Aeroelectronics 

368  CS —  Computor  Science 

600  DC —  Electrical  Engineering  (Comms) 

Note:  DCTR  is  curriculum  460 

590  EC —  Electrical  Engineering  ;  various 

ED —  sub-specialties.  Note:  EETR  is 

EG —  curriculum  460 

ER — 


380 


GM — 


Nath  option  of  advanced  science 


i  mill  i.  i, 


Specialt  y 
Curricula  Degree 
Numbers  Codes  Program 

****************************************************** 


38  1 

GP — 

Physics  option  of  adv.  science 

< 

620 

i 

• 

HC — 

Communications  Management 

HM — 

525 

IX — 

Antisubmarine  Warfare  Note:  IETR 

is  curriculum  460 

4 

•  | 

365 

JM — 

Command,  Control  and  Communica¬ 

1 

t 

JN — 

tions  ;  various  sub-specialties 

J 

JO — 

Note:  JCTE 

and  JXTR  are  from 

i 

• 

JP — 

curricula  460 

1 

JS — 

595 

KE — 

Electronic 

Warfare  Systems  Tech- 

nology  Note:  KETR  is  curric  460 

-* 

• 

817 

HE — 

Admin  Sci  - 

Econ,  JSHC 

837 

HF — 

Admin  Sci  - 

Financial  mgt 

•4 

857 

MH — 

Admin  Sci  - 

Human  Resource  mgt 

• 

819 

HI  — 

Admin  Sci  - 

Inv  entory 

817 

MK — 

Admin  Sci  - 

International  stu. 

• 

827 

MM — 

Admin  Sci  - 

Material 

817 

MO — 

Admin  Sci  - 

Ops  Research,  Army 

847 

MP — 

Admin  Sci  - 

Personnel 

• 

815 

MP  — 

Admin  Sci  - 

Procurement 

817 

MS — 

Admin  Sci  - 

MGMT  Sci,  Coast  Guard 

814 

HT — 

Admin  Sci  - 

Transportation 
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Specialt  v 
Curricula  Degree 
Numbers  Codes  Program 

********************$********************************* 


c 


m 


8  1  3 

MV — 

Admin  Sci  -  Material  Movement 

817 

MX — 

Admin  Sci  -  Civ.  Federal  Program 

570 

NE — 

Mechanical  Engineering 

NF — 

ME  via  460 

NI — 

ME  via  460 

NR — 

ME  via  460 

NT — 

ME  via  460 

441 

OH — 

Hydrographic  Science 

440 

OS  — 

Oceanography  Note:  OCTR  is  460 

367 

PL — 

Information  Systems  (from  CS ) 

360 

RA — 

Cps  Analysis,  Army 

460 

RC — 

Engr  Sci  for  Ops  Analysis 

360 

RG — 

Cps  Analysis,  General 

KN — 

Navy 

RS — 

Supply 

RP — 

PHD  program 

591 

SE — 

Space  Systems  Engineering 

366 

SO — 

Space  Systems  Operations  Note: 

SXTR  and  SOTR  are  curriculum  460 

535 


UX — 


Underwater  Acoustics  Note:  UETR 
is  curriculum  460 


Specialty 
Curricula  Degree 
Numbers  Codes  Program 

******************************  ***$*********  ******* 


530 

PA¬ 
RC — 

WE — 

WM — 

Weapons  Systems 

Technology 

532 

WN — 

Weapons  Systems 

Science 

530 

PP — 

Weapons  Systems 

Technology 

531 

ws — 

Weapons  Systems 

Science 

372 

XA — 

Meteorology 

373 

XS — 

Met.  and  Oceanography 

374 

XT — 

Air-Ocean  Tactical  Env.  Suppo 

Note:  XOTR  and  XXTR  are  460 


683 

YA — 

NSA 

— 

USSR,  Europe;  USN 

682 

YB — 

NS  A 

- 

Asia/Pacific;  USN 

681 

YC — 

NSA 

- 

Middle  East,  Africa;  US 

YD — 

- 

S.  Asia;  USAF,  USA 

682 

YE— 

NSA 

- 

Asia/Pacific;  USAF,  USA 

68  1 

YF — 

NSA 

- 

S.  Africa;  USAF,  USA 

YG — 

- 

N.  Africa;  USAF,  USA 

Yfl — 

- 

Middle  East;  USAF,  USA 

687 

YN — 

NSA 

- 

Nuclr/Plng;  USN 

684 

YO — 

NSA 

- 

Intl/Org/Neg ;  USN 
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Specialty 
Curricula  Degree 
Numbers  Codes  Program 

****************************************************** 


686 

YP — 

NSA 

-  St rat/Plng;  USN 

683 

YR — 

NS  A 

-  DSSR/E. 

Europe 

825 

YS — 

NSA 

-  Navy.  Intell;  LIS N 

683 

YW — 

NSA 

-  USSR/K. 

Europe 
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APPENDIX  C 

DETAILED  SUMMARY  OF  RESULTS 


This  appendix  contains  a  brief  description  of  the 
results  found  for  each  course  and  curriculum.  These  results 
are  listed  in  ascending  numerical  order  first  for  the 
courses  then  the  curricula.  Courses  or  curricula  that  cause 
problems  are  flagged  with  a  'P'  in  front  of  the  identifying 
number. 

Course 

Number  Comments 

******  **************************************** *********** 
P  2101  Not  taught,  only  predicted. 

P  2102  Large  magnitude  errors,  generally  underestimates. 

Curriculum  590  causes  most  of  the  problems,  curric¬ 
ulum  600  to  a  lesser  extent.  No  quarterly  pattern. 

P  2103  Generally  gocd  except  for  curriculum  531  which 

never  sends  people  but  has  predicted  40,  27  and  7. 
No  significant  quarterly  pattern. 

P  3001  Prediction  errors  in  third  and  fourth  quarters  - 
not  predicted  but  taught.  Only  curriculum  368. 

3002  Generally  gocd.  Only  taken  by  curriculum  825  though 
others  are  predicted.  Need  more  data. 

3003  Generally  good.  Need  more  data. 

3004  Generally  good. 


3005  Generally  good.  Need  more  data. 


Course 

Number 

****** 

3006 

P  3008 

P  3101 

P  3102 

P  3103 

P  3104 


P  310  5 

3302 

3303 

3401 

3402 


Comments 

*************************************************** 

Generally  gccd  except  for  curriculum  817.  Only 
taught  first  and  third  guarters.  Overall  class 
sizes  fairly  stable  at  63  and  87  respectively 
(three  sections  required)  . 

Definite  prediction  problem  that  appears  to  be 
improving.  Curriculum  365  provides  most  of  the 
input. 

Curriculum  460  not  predicted  generally  causing  a 
significant  underestimation  (usually  results  in 
adding  an  additional  section) .  More  data  needed. 

Overall  significant  overestimation  from  cumulative 
effect  of  8 XX  curricula.  Worst  in  third  quarter. 

Overall  significant  over estimation  from  cumulative 
effect  of  8XX  curricula.  No  quarterly  trend. 

Large  errors  in  summary  predictions  generally  due 
to  prediction  vice  accuracy  errors.  All  curricula 
except  570  and  817  experience  these  errors.  First 
quarter  predictions  look  good,  underestimates  occur 
in  the  second  quarter,  overestimates  in  the  third. 

Significant  prediction  problems.  Generally  offered 
only  third  guarter. 

Not  enough  data. 

Generally  good.  525  provides  majority  of  input. 

Not  enough  data. 

Prediction  problem,  not  severe. 
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Number  Comments 

******  *************************************************** 

3403  Generally  excellent,  not  enough  data. 

P  3404  Curricula  365  and  825  provide  most  of  the  inputs 
to  this  course.  365  does  an  excellent  job,  825  a 
poor  one. 

3601  Generally  gccd. 

3602  Generally  excellent. 

3603  Generally  excellent. 

3604  Generally  good. 

P  3702  Significant  prediction  problem.  Not  taught  first 
quarter,  only  predicted.  Curriculum  847  provides 
most  of  the  input. 

4601  Generally  excellent. 

4602  Generally  excellent,  needs  more  data. 

4701  Generally  gocd. 

*************************************************  *********** 
**  END  OF  COURSE  LISTING  ** 

************************************  ************************ 


Curriculum 

Number  Comments 

******  ************************************************* 

P  0  Summary  predictions  are  innaccurate  due  to  course 

effects.  No  quarterly  trend  seen. 

p  360  Seldom  predicted  for  OS  courses.  Minor  effect  due 

to  small  number  of  inputs. 

365  Generally  excellent,  possible  problem  with  OS3008 

366  Generally  excellent. 

367  Generally  good,  need  more  data  in  OS3101. 

P  368  Third  guarter  prediction  problem  developing. 

372  Not  enough  data. 

373  Not  enough  data. 

P  374  Tends  to  overestimate. 

380  Not  enough  data. 

440  Not  enough  data. 

441  Not  enough  data. 

P  460  Never  predicted. 

525  Generally  excellent. 

P  530  Problem  in  OS3104. 

P  531  Significant  problems  affecting  OS2103  and  0S3104. 

532  Not  enough  data. 

535  Not  enough  data. 


570 


Possible  errors  in  third  guarter,  need  more  data. 


Curr iculum 

Number  Comments 

#******t***$*****$****$*****#*:fr***:(t$**<C  #$*$*$*$*** 


P 


P 

P 

P 

P 

P 

P 

P 


590 

Problem  in  OS2102 

591 

Looks  good,  not  enough  data. 

• 

600 

Generally  good. 

610 

Not  enough  data. 

611 

Not  enough  data. 

• 

620 

Generally  good. 

683 

Mot  enough  data. 

• 

686 

Not  enough  data. 

687 

Not  enough  data. 

813 

Generally  excellent. 

814 

Generally  excellent. 

815 

Generally  fair. 

817 

Problem  with  OS3006,  good  otherwise. 

• 

819 

Various  prediction  problems. 

825 

Problem  with  OS3404. 

• 

327 

Overestimates  for  OS3102  and  OS3103. 

837 

Problem  in  OS3102,  prediction  error  for  OS3105. 

847 

Problem  with  OS3702.  Good  with  OS3103.  Third 

• 

quarter  predictions  are  poor. 

857 

Routine  innaccuracies  with  no  apparent  pattern. 

999 

Insignificant  input. 

*."  '.  '.*  .*  ‘  ?*t\  •' .  *'.  *"- 
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APPENDIX  D 

CONDENSED  COOBSE  MATRICES 


This  appendix  contains  the  condensed  course  matrices  for 
all  curricula  that  take  OS  courses.  Curricula  610,  611,  681, 
682,  683,  and  684  dc  not  have  any  OS  courses  required  and 
are  not  included,  1 xxxxxx’  entries  in  the  condensed  matrices 
indicate  that  the  students  are  onboard  but  not  taking  OS  or 
OA  designated  courses  (these  entries  are  not  made  following 
the  final  OS  or  OA  course  taken  to  prevent  display  clutter). 
The  entries  are  based  on  the  printed  course  matrices  as 
published  as  of  August,  1984.  These  matrices  were  provided 
by  the  individual  curricular  offices. 
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Table  9 

Condensed  Coarse  Matrices  (continued) 


Condensed  Coarse  Matrices  (continued) 
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APPENDIX  E 

ENTRY  /  TRACK  COORSE  DESIGNATIONS 

This  appendix  contains  a  complete  listing  of  the  OS 
courses  divided  into  ’Entry'  and  'Follow-on'  catagories.  An 
'Entry'  course  is  the  first  OS  course  that  is  taken  in  a 
given  course  matrix  (for  a  given  curriculum)  ;  the  first  time 
a  curriculum  can  te  'seen'  by  the  Operations  Analysis 
Department.  ’Follov-cn'  courses  are  those  courses  that  are 
taken  in  quarters  following  the  quarter  the  'Entry'  course 
is  taken.  All  curricula  take  one  of  the  six  'Entry'  courses 
prior  to  any  further  OS  course  work  with  the  exceptions  of 
curricula  610,  611,  681,  632,  683  and  684  which  have  no 

required  OS  courses. 

ENTRY  TRACK  (Follov-on) 

COURSE  CURRICULUM  COURSES,  ALL  OS 
******  **********  ******************************* 

OS2103  365  3604,3404,3008,3603,4602 

366  3604,3404,3008,3603,4602 

374  3601,3602,3603,3604 

525  3303,3604,3601,3401,4601,3602 

535  none 

595  3604,3003,3603,3403,4601 

CS3101  367  3004 

620  3005 

686  none 
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ENTRY 

COURSE 

*  * 

CS3 10  1 


OS3001 

OS3  104 


CS2102 


CS3102 


TRACK  (Follow-on) 

CURRICULUM  COURSES,  ALL  OS 
**********  ******************************* 


687  none 

825  3404,3002 


368  none 


372 

373 

440 

441 

530 

531 

532 
570 


none 

none 

none 

none 

none 

none 

none 

none 


590  3104 

591  none 

600  none 


813  3103,3006 

814  3103,3006 

815  3103,3006 


817 


3103, 3006,3604 


f' 


r* 


ENTRY 

COURSE 

****** 

CURRICULUM 

********** 

TFACK  (Follow-on) 

CCURSES,  ALL  OS 

******************************* 

OS3  102 

819 

3103,  3006 

827 

8103,3006 

837 

3103,3006 

847 

3103,3006,3702 

857 

2103,3006 

r— * 

a  ■ 


s» 
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